CN111160367B - Image classification method, apparatus, computer device, and readable storage medium - Google Patents
Image classification method, apparatus, computer device, and readable storage medium Download PDFInfo
- Publication number
- CN111160367B CN111160367B CN201911338271.7A CN201911338271A CN111160367B CN 111160367 B CN111160367 B CN 111160367B CN 201911338271 A CN201911338271 A CN 201911338271A CN 111160367 B CN111160367 B CN 111160367B
- Authority
- CN
- China
- Prior art keywords
- image
- image classification
- result
- medical image
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000011218 segmentation Effects 0.000 claims abstract description 71
- 238000012549 training Methods 0.000 claims abstract description 53
- 238000001514 detection method Methods 0.000 claims description 40
- 238000004590 computer program Methods 0.000 claims description 24
- 238000010586 diagram Methods 0.000 claims description 9
- 238000003384 imaging method Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 abstract description 18
- 238000012790 confirmation Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 description 6
- 210000003857 wrist joint Anatomy 0.000 description 5
- 238000002591 computed tomography Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 210000001015 abdomen Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003902 lesion Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000003187 abdominal effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000001145 finger joint Anatomy 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 208000003174 Brain Neoplasms Diseases 0.000 description 1
- 206010008111 Cerebral haemorrhage Diseases 0.000 description 1
- 206010014561 Emphysema Diseases 0.000 description 1
- 208000019693 Lung disease Diseases 0.000 description 1
- 206010056342 Pulmonary mass Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 210000000629 knee joint Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013421 nuclear magnetic resonance imaging Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The application relates to an image classification method, an image classification device, a computer device and a readable storage medium. The method comprises the following steps: acquiring a medical image to be classified; inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels; and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part. In the method, the computer equipment adopts a multi-task network model to determine the shooting position in the medical image, so that the accuracy of the result of the target shooting position can be improved, and the accuracy of the subsequent focus recognition process is further improved; and the method does not need the radiologist to make a confirmation process, and improves the efficiency of the focus identification process.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image classification method, an image classification apparatus, a computer device, and a readable storage medium.
Background
X-ray films (X-Rays) are important in early detection of pulmonary diseases, heart diseases, abdominal diseases, fractures, and the like, due to their low price and good imaging effect. After a scanning technician finishes scanning a patient by using medical equipment, submitting a medical image to a corresponding radiologist for reading, and identifying focus features in the medical image by the radiologist according to self experience to give a focus identification result.
Typically, when a scanning technician scans a portion of a patient, the scanned portion label (e.g., the current medical image is an abdominal image) is entered, and the radiologist then performs lesion recognition on the medical image based on the portion label.
However, in the conventional technology, the position label is wrongly recorded due to the error of the scanning technician, which requires the radiologist to judge the scanning position first and then identify the focus, and the efficiency and accuracy of the focus identification process are low.
Disclosure of Invention
Based on this, it is necessary to provide an image classification method, apparatus, computer device and readable storage medium for the problem that the efficiency and accuracy of the lesion recognition process in the conventional art are low.
In a first aspect, an embodiment of the present application provides an image classification method, including:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the training mode of the multi-task network model comprises the following steps:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
training the initial multi-task network according to the first loss and the second loss to obtain a multi-task network model.
In one embodiment, inputting a medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image, including:
extracting features of the medical image by adopting a first convolution layer in the multitasking network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of key points;
and carrying out feature classification on the feature map by adopting a pooling layer and a full-connection layer in the multi-task network model to obtain an image classification result.
In one embodiment, when the segmentation result and the image classification result of the key point meet the preset conditions, determining the shooting location represented by the image classification result as the target shooting location includes:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the preset corresponding relation between the number of key points and the image classification;
if yes, determining the shooting part represented by the image classification result as a target shooting part.
In one embodiment, the target capture location includes at least one of a location code, a location name, and a location orientation, the location orientation including a normal or side position.
In one embodiment, after determining the shooting location characterized by the image classification result as the target shooting location, the method further includes:
acquiring a shooting tag of a medical image, wherein the shooting tag is shooting position data recorded by a user when shooting the medical image;
if the shooting tag is inconsistent with the target shooting position, updating the shooting tag to the target shooting position.
In one embodiment, after determining the shooting location characterized by the image classification result as the target shooting location, the method further includes:
determining a focus detection algorithm corresponding to the target shooting position according to the target shooting position of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
In a second aspect, an embodiment of the present application provides an image classification apparatus, including:
the acquisition module is used for acquiring medical images to be classified;
the processing module is used for inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and the determining module is used for determining the shooting part represented by the image classification result as a target shooting part when the segmentation result and the image classification result of the key points meet the preset conditions.
In a third aspect, embodiments of the present application provide a computer device, including a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
The image classification method, the device, the computer equipment and the readable storage medium can acquire medical images to be classified; inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels; and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part. In the method, the computer equipment adopts a multi-task network model to determine the shooting position in the medical image, so that the accuracy of the result of the target shooting position can be improved, and the accuracy of the subsequent focus recognition process is further improved; and the method does not need the radiologist to make a confirmation process, and improves the efficiency of the focus identification process.
Drawings
FIG. 1 is a flow chart of an image classification method according to an embodiment;
FIG. 1a is a schematic diagram of a multi-task network model according to one embodiment;
FIG. 1b is a schematic diagram of a correspondence between the number of keypoints and image categories according to one embodiment;
FIG. 1c is a schematic diagram of a target capture site and a corresponding medical image according to one embodiment;
FIG. 2 is a flow chart of an image classification method according to another embodiment;
FIG. 3 is a flow chart of an image classification method according to another embodiment;
FIG. 4 is a flow chart of an image classification method according to another embodiment;
FIG. 5 is a schematic diagram of an image classification apparatus according to an embodiment;
FIG. 6 is a schematic diagram of an image classification apparatus according to another embodiment;
fig. 7 is a schematic diagram of an internal structure of a computer device according to an embodiment.
Detailed Description
The image classification method provided by the embodiment of the application can be suitable for classifying the shot medical image so as to determine the shooting position of the medical image. The medical image may be an X-ray film, an electronic computed tomography image (Computed Tomography, CT), a magnetic resonance image (Nuclear Magnetic Resonance Imaging, MRI) or a positron emission computed tomography image (Positron Emission Computed Tomography, PET), etc. In the traditional technology, a scanning technician usually records a position label of a medical image, a radiologist carries out focus identification on the medical image according to the position label, but the situation that the scanning technician wrongly records the position label occurs, so that the radiologist is required to carry out re-judgment on the scanning position and focus identification, and the efficiency and the accuracy of the focus identification process are low. The embodiment of the application provides an image classification method, an image classification device, computer equipment and a readable storage medium, which aim to solve the technical problems.
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application are further described in detail by the following embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The execution subject of the method embodiments described below may be an image classification apparatus, which may be implemented as part or all of a computer device by software, hardware, or a combination of software and hardware. The following method embodiments are described taking an execution subject as an example of a computer device, where the computer device may be a terminal, a server, a separate computing device, or integrated on a medical imaging device, and this embodiment is not limited to this.
Fig. 1 is a flow chart of an image classification method according to an embodiment. The embodiment relates to a specific process that computer equipment judges medical images to be classified to obtain target shooting parts of the medical images. As shown in fig. 1, the method includes:
s101, acquiring medical images to be classified.
In particular, the medical image to be classified is a captured image of a portion of the patient, such as an X-ray image, which may be acquired by a computer device from a post-processing workstation or a picture archiving and communication system (Picture Archiving and Communication Systems, PACS). Alternatively, the computer device may acquire medical images uploaded to the PACS system by the radiologist in real time, or may acquire all medical images in this time period from the PACS system at regular time intervals. Optionally, the computer device may also acquire medical images to be classified from a hospital information management system (Hospital Information System, HIS), a clinical information system (Clinical Information System, CIS), a radiology information management system (Radiology Information System, RIS), an electronic medical record system (Electronic Medical Record, EMR), and a related medical image cloud storage platform.
S102, inputting a medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels.
Specifically, the computer device may input the acquired medical image to be classified into a preset multi-task network model, where the multi-task network model may simultaneously implement the image segmentation and image classification processes, so as to obtain a segmentation result and an image classification result of the key points in the medical image. The key points in the medical image are set according to the characteristics of different parts of a patient, for example, the wrist joint image comprises 5 key points, and correspondingly, 5 fingers; the image classification result may characterize the photographed site. The multi-task network model can be obtained by training according to training samples with key point labels and classification labels, and therefore the medical image can be accurately segmented and classified by the multi-task network model obtained by training. Alternatively, the segmentation result of the keypoints may include the position, number, and segmented keypoint images of the keypoints.
Optionally, the computer device may also perform data enhancement (Data Augamentation) on the medical image prior to inputting the medical image into the multitasking network model: carrying out random left-right horizontal overturn, horizontal translation transformation in the horizontal direction and the vertical direction, random rotation, filling of edges and contrast change on the medical image; and then carrying out normalization and normalization preprocessing operation on the medical image to obtain a normalized image which is input into the multi-task network model.
Alternatively, the multi-task network model may be a multi-layer convolutional neural network, a network model formed by combining a segmentation network and a classification network, or other deep learning networks, which is not limited in this embodiment. When the multi-task network model is a multi-layer convolutional neural network, the network structure can be as shown in fig. 1a, the first half part of the network is an extracted feature, the second half part is two branches, the first branch mainly carries out key point detection through segmentation, the second branch mainly classifies the categories of the images, and finally the segmentation result and the image classification result of the key points are output.
And S103, when the segmentation result and the image classification result of the key points meet preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
Specifically, when the segmentation result and the image classification result of the key points meet the preset conditions, the computer device may determine that the shooting location represented by the image classification result is a target shooting location of the medical image. Alternatively, the preset condition may be a correspondence between the number of keypoints and the image category, where the correspondence may be as shown in fig. 1b, where, for example, the wrist positive image corresponds to 5 keypoints and the chest side corresponds to 10 keypoints, and only when the obtained keypoint segmentation result and the image classification result satisfy the correspondence, the target shooting location may be determined. Optionally, the preset condition may be a correspondence between the positions of the key points and the image types, for example, the positions of the finger joints of the 5 fingers corresponding to the wrist joint normal position image may be determined to be the wrist joint normal position only when the obtained segmentation result of the key points is the finger joint positions of the 5 fingers and the image classification result is the wrist joint normal position image.
Optionally, the target shooting location includes at least one of a location code, a location name, and a location orientation, where the location codes of different shooting locations are different, such as a wrist joint of 0, a chest of 1, an abdomen of 2, a head of 3, and the like, and the location orientation includes a normal position or a lateral position. Wherein, the target shooting part and the corresponding medical image can be seen in fig. 1 c.
Further, after the computer device determines the target shooting position, the target shooting position can be displayed to the radiologist, so that the radiologist can recognize the focus according to the target shooting position, and the progress of focus recognition can be quickened.
According to the image classification method provided by the embodiment, the computer equipment inputs the acquired medical images to be classified into the multi-task network model to obtain the segmentation results and the image classification results of the key points in the medical images, and when the segmentation results and the image classification results of the key points meet preset conditions, the shooting parts represented by the image classification results can be determined to be target shooting parts. In the method, the computer equipment adopts a multi-task network model to determine the shooting position in the medical image, so that the accuracy of the result of the target shooting position can be improved, and the accuracy of the subsequent focus recognition process is further improved; and the method does not need the radiologist to make a confirmation process, and improves the efficiency of the focus identification process.
In some embodiments, the training samples include a plurality of sample images and labels corresponding to each sample image, where the labels include a keypoint label and a classification label; as shown in fig. 2, the training manner of the multitasking network model includes:
s201, inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result.
Specifically, the initial multi-task network may be a newly built network, and the computer device inputs the sample image into the initial multi-task network, so as to obtain an initial key point segmentation result and an initial image classification result. Since the precision of the initial multitasking network is not yet very high at this point, we will output the result as the initial result. Optionally, the initial keypoint segmentation result may also include the position and number of the initial keypoints and the segmented keypoint images, and the initial image classification result is a shooting position in the sample image. The initial multi-task network can be divided into two parts, the first half part of the network is used for extracting features, the second half part of the network is used for two branches, the first branch mainly carries out key point detection through segmentation, and the second branch mainly classifies the categories of the images.
Optionally, the computer device may also perform data enhancement (Data Augamentation) on the sample image prior to entering the sample image into the initial multitasking network: carrying out random left-right horizontal overturn, horizontal translation transformation in the horizontal direction and the vertical direction, random rotation, filling of edges and contrast change on the sample image; and then carrying out normalization and normalization preprocessing operation on the sample image to obtain a normalized image, and inputting the normalized image into an initial multi-task network.
S202, calculating a first loss between an initial key point segmentation result and a key point label and a second loss between an initial image classification result and a classification label.
Specifically, each sample image may be marked in advance by an experienced doctor, that is, the positions, the number, and the key point images of the key points, and the category of the sample image are marked as key point tags and category tags. The computer device then calculates a first loss between the resulting initial keypoint segmentation and the keypoint label and a second loss between the resulting initial image classification and the classification label.
Alternatively, the computer device may calculate the first loss and the second loss using a cross entropy loss function, or may calculate the losses using other types of loss functions, which is not limited in this embodiment.
And S203, training the initial multi-task network according to the first loss and the second loss to obtain a multi-task network model.
In particular, the computer device may train the initial multi-tasking network according to the first loss and the second loss, i.e. adjust network parameters of the initial multi-tasking network according to the first loss and the second loss. Alternatively, the computer device may add the first loss and the second loss, or weight the sum, or average the sum, resulting in an overall loss to adjust network parameters of the initial multi-tasking network. And when the overall loss is smaller than or equal to a preset threshold value or convergence is reached, the initial multi-task network training is represented to be completed, and a multi-task network model is obtained.
Optionally, in the training sample, a part of the training set may be selected as the training set, and another part of the training set may be selected as the testing set, where after the training set completes training the initial multi-task network, the computer device may further test the trained network using the testing set, so as to further ensure accuracy of the multi-task network model.
According to the image classification method provided by the embodiment, the computer equipment trains the initial multi-task network by using the training sample, so that the multi-task network model with higher precision can be obtained through iterative training, the accuracy of the obtained key point segmentation result and the image classification result is improved, and the accuracy of the focus recognition process is further improved.
As shown in fig. 1a, the first half of the multi-task network model is feature extraction, and the second half is keypoint detection and classification, and then the step S102 may include: extracting features of the medical image by adopting a first convolution layer in the multitasking network model to obtain a feature map of the medical image; performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of key points; and carrying out feature classification on the feature map by adopting a pooling layer and a full-connection layer in the multi-task network model to obtain an image classification result.
The feature image of the medical image can be obtained by convolving a plurality of convolution layers of the multi-task network model, and then the feature image is subjected to key point feature detection by a plurality of other convolution layers, so that a key point segmentation result can be obtained, namely, a key point image and a background image are distinguished; and mapping and classifying the features in the feature map through the pooling layer and the full connection layer to obtain an image classification result.
Optionally, in some embodiments, S103 may include: determining the number of key points according to the segmentation result of the key points; judging whether the number of key points and the image classification result meet the preset corresponding relation between the number of key points and the image classification; if yes, determining the shooting part represented by the image classification result as a target shooting part.
When the segmentation result of the key points only includes the positions of the key points or the segmented images, the computer device may determine the number of the key points according to the number of the positions or the number of the segmented images, and then determine whether the number of the key points and the image classification result determined by the multi-task network model satisfy the corresponding relationship according to the corresponding relationship between the preset number of the key points and the image category, as shown in fig. 1 b. If yes, taking the shooting part represented by the image classification result as a target shooting part.
Optionally, in some embodiments, as shown in fig. 3, the method further includes:
s301, acquiring a shooting label of a medical image, wherein the shooting label is shooting position data recorded by a user when the medical image is shot.
S302, if the shooting tag is inconsistent with the target shooting position, updating the shooting tag to the target shooting position.
In particular, the computer device may obtain a photographing tag carried by the medical image, where the photographing tag is recorded by a scanning technician when photographing the medical image, such as a chest tag, a head tag, an abdomen tag, and the like. And the computer equipment judges whether the shooting label input by the scanning technician is consistent with the obtained target shooting position, and if the shooting label is inconsistent with the obtained target shooting position, the computer equipment updates the shooting label into the target shooting label. For example, if the imaging label entered by the scanning technician is a chest label and the obtained target imaging site is abdomen-positioned, and the two are not identical, the computer device updates the chest label to an abdomen label if the entering by the scanning technician is considered to be incorrect. Therefore, timely correction can be given when related data of the medical image are wrong, so that a data basis is made for subsequent other data analysis, and other data analysis processes are not influenced.
Optionally, in some embodiments, as shown in fig. 4, after determining the target capturing location, the method further includes:
s401, determining a focus detection algorithm corresponding to the target shooting position according to the target shooting position of the medical image.
S402, detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
Specifically, the computer device may determine a focus detection algorithm corresponding to the target shooting location according to the obtained target shooting location, where the focus detection algorithm may be stored in an algorithm library of the computer device. If the target shooting part is lung positive, a lung nodule detection algorithm, an emphysema detection algorithm and the like can be called to detect focus; when the target shooting part is a head positive, a cerebral hemorrhage detection algorithm, a brain tumor detection algorithm and the like can be called to detect focus; when the target shooting position is the side position of the knee joint, a fracture detection algorithm can be called to detect the target shooting position, and the like. Alternatively, the above-mentioned lesion detection algorithm may be a neural network algorithm, or may be another type of algorithm, which is not limited in this embodiment.
According to the image classification method, after the shooting position of the medical image is judged by using the computer equipment, a corresponding focus detection algorithm can be automatically invoked to detect the medical image, and a focus detection result is obtained. The full automation of the focus identification process can be realized, the human participation is not needed, and the efficiency and the accuracy of the focus identification process can be further improved.
It should be understood that, although the steps in the flowcharts of fig. 1-4 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 1-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, nor does the order in which the sub-steps or stages are performed necessarily occur in sequence, but may be performed alternately or alternately with at least a portion of the other steps or sub-steps or stages of other steps.
Fig. 5 is a schematic structural diagram of an image classification apparatus according to an embodiment. As shown in fig. 5, the apparatus includes: an acquisition module 11, a processing module 12 and a determination module 13.
Specifically, the acquiring module 11 is configured to acquire a medical image to be classified.
The processing module 12 is used for inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels.
And the determining module 13 is configured to determine the shooting location represented by the image classification result as the target shooting location when the segmentation result and the image classification result of the key point meet the preset condition.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; as shown in fig. 6, the apparatus further includes a training module 14.
Specifically, the training module 14 is configured to input the sample image into an initial multi-task network to obtain an initial keypoint segmentation result and an initial image classification result; calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label; training the initial multi-task network according to the first loss and the second loss to obtain a multi-task network model.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
In one embodiment, the processing module 12 is specifically configured to perform feature extraction on the medical image by using a first convolution layer in the multitasking network model to obtain a feature map of the medical image; performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of key points; and carrying out feature classification on the feature map by adopting a pooling layer and a full-connection layer in the multi-task network model to obtain an image classification result.
In one embodiment, the determining module 13 is specifically configured to determine the number of keypoints according to the segmentation result of the keypoints; judging whether the number of key points and the image classification result meet the preset corresponding relation between the number of key points and the image classification; if yes, determining the shooting part represented by the image classification result as a target shooting part.
In one embodiment, the target capture location includes at least one of a location code, a location name, and a location orientation, the location orientation including a normal or side position.
In one embodiment, the apparatus further includes an update module; the acquiring module 11 is further configured to acquire a shooting tag of the medical image, where the shooting tag is shooting position data recorded by a user when the medical image is shot; and the updating module is used for updating the shooting tag to the target shooting position if the shooting tag is inconsistent with the target shooting position.
In one embodiment, the apparatus further includes a detection module configured to determine a focus detection algorithm corresponding to the target shooting location according to the target shooting location of the medical image; and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
For specific limitations of the image classification apparatus, reference may be made to the above limitations of the image classification method, and no further description is given here. The respective modules in the above-described image classification apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image classification method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 7 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
The computer device provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiment, and will not be described herein.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the processor when executing the computer program also implements the steps of:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
training the initial multi-task network according to the first loss and the second loss to obtain a multi-task network model.
In one embodiment, the processor when executing the computer program further performs the steps of:
extracting features of the medical image by adopting a first convolution layer in the multitasking network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of key points;
and carrying out feature classification on the feature map by adopting a pooling layer and a full-connection layer in the multi-task network model to obtain an image classification result.
In one embodiment, the processor when executing the computer program further performs the steps of:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the preset corresponding relation between the number of key points and the image classification;
if yes, determining the shooting part represented by the image classification result as a target shooting part.
In one embodiment, the target capture location includes at least one of a location code, a location name, and a location orientation, the location orientation including a normal or a side.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a shooting tag of a medical image, wherein the shooting tag is shooting position data recorded by a user when shooting the medical image;
if the shooting tag is inconsistent with the target shooting position, updating the shooting tag to the target shooting position.
In one embodiment, the processor when executing the computer program further performs the steps of:
determining a focus detection algorithm corresponding to the target shooting position according to the target shooting position of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels;
and when the segmentation result and the image classification result of the key points meet the preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
The computer readable storage medium provided in this embodiment has similar principles and technical effects to those of the above method embodiment, and will not be described herein.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the computer program when executed by the processor also performs the steps of:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
training the initial multi-task network according to the first loss and the second loss to obtain a multi-task network model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
extracting features of the medical image by adopting a first convolution layer in the multitasking network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of key points;
and carrying out feature classification on the feature map by adopting a pooling layer and a full-connection layer in the multi-task network model to obtain an image classification result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the preset corresponding relation between the number of key points and the image classification;
if yes, determining the shooting part represented by the image classification result as a target shooting part.
In one embodiment, the target capture location includes at least one of a location code, a location name, and a location orientation, the location orientation including a normal or a side.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a shooting tag of a medical image, wherein the shooting tag is shooting position data recorded by a user when shooting the medical image;
if the shooting tag is inconsistent with the target shooting position, updating the shooting tag to the target shooting position.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining a focus detection algorithm corresponding to the target shooting position according to the target shooting position of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. An image classification method, comprising:
acquiring a medical image to be classified;
inputting the medical image into a preset multitasking network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels; the segmentation result of the key points is used for obtaining the number of the key points;
when the segmentation result of the key points and the image classification result meet preset conditions, determining a shooting part represented by the image classification result as a target shooting part; the preset conditions comprise the corresponding relation between the preset key point number and the image category.
2. The method of claim 1, wherein the training sample comprises a plurality of sample images and a label corresponding to each sample image, the label comprising a keypoint label and a classification label; the training mode of the multi-task network model comprises the following steps:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
training the initial multi-task network according to the first loss and the second loss to obtain the multi-task network model.
3. The method according to claim 1 or 2, wherein the inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of a key point in the medical image comprises:
extracting the characteristics of the medical image by adopting a first convolution layer in the multitasking network model to obtain a characteristic diagram of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multi-task network model to obtain a segmentation result of the key points;
and adopting a pooling layer and a full-connection layer in the multi-task network model to perform feature classification on the feature map to obtain the image classification result.
4. The method according to claim 1 or 2, wherein when the segmentation result of the keypoints and the image classification result satisfy a preset condition, determining the shooting location represented by the image classification result as the target shooting location includes:
determining the number of the key points according to the segmentation result of the key points;
judging whether the number of the key points and the image classification result meet the preset corresponding relation between the number of the key points and the image classification;
if yes, determining the shooting part represented by the image classification result as a target shooting part.
5. The method of claim 4, wherein the target capture location comprises at least one of a location code, a location name, and a location orientation, the location orientation comprising a normal or a side.
6. The method of claim 1, wherein after determining the imaging location characterized by the image classification result as the target imaging location, the method further comprises:
acquiring a shooting tag of the medical image, wherein the shooting tag is shooting position data recorded by a user when shooting the medical image;
and if the shooting label is inconsistent with the target shooting position, updating the shooting label to the target shooting position.
7. The method of claim 1, wherein after determining the imaging location characterized by the image classification result as the target imaging location, the method further comprises:
determining a focus detection algorithm corresponding to a target shooting position according to the target shooting position of the medical image;
and detecting the medical image according to the focus detection algorithm to obtain a focus detection result.
8. An image classification apparatus, comprising:
the acquisition module is used for acquiring medical images to be classified;
the processing module is used for inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to training samples with key point labels and classification labels; the segmentation result of the key points is used for obtaining the number of the key points;
the determining module is used for determining that the shooting part represented by the image classification result is a target shooting part when the segmentation result of the key points and the image classification result meet preset conditions; the preset conditions comprise the corresponding relation between the preset key point number and the image category.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338271.7A CN111160367B (en) | 2019-12-23 | 2019-12-23 | Image classification method, apparatus, computer device, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338271.7A CN111160367B (en) | 2019-12-23 | 2019-12-23 | Image classification method, apparatus, computer device, and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111160367A CN111160367A (en) | 2020-05-15 |
| CN111160367B true CN111160367B (en) | 2024-03-01 |
Family
ID=70557819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911338271.7A Active CN111160367B (en) | 2019-12-23 | 2019-12-23 | Image classification method, apparatus, computer device, and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111160367B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696083B (en) * | 2020-05-20 | 2024-05-14 | 平安科技（深圳）有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN111640093A (en) * | 2020-05-20 | 2020-09-08 | 上海联影智能医疗科技有限公司 | Quality control method for medical image and computer-readable storage medium |
| CN111652300A (en) * | 2020-05-27 | 2020-09-11 | 联影智能医疗科技(北京)有限公司 | Spine curvature classification method, computer device and storage medium |
| CN111932547B (en) * | 2020-09-24 | 2021-06-11 | 平安科技（深圳）有限公司 | Method and device for segmenting target object in image, electronic device and storage medium |
| TWI812888B (en) | 2020-10-14 | 2023-08-21 | 財團法人工業技術研究院 | Image recognition method and image recognition system |
| CN113140290A (en) * | 2020-12-17 | 2021-07-20 | 慧影医疗科技（北京）有限公司 | Model training method and device, focus identification method and electronic equipment |
| CN112766314B (en) * | 2020-12-31 | 2024-05-28 | 上海联影智能医疗科技有限公司 | Anatomical structure recognition method, electronic device, and storage medium |
| CN115049580A (en) * | 2021-03-09 | 2022-09-13 | 杭州朝厚信息科技有限公司 | Method for detecting key point of X-ray head photographic image |
| CN112802036A (en) * | 2021-03-16 | 2021-05-14 | 上海联影医疗科技股份有限公司 | Method, system and device for segmenting target area of three-dimensional medical image |
| CN113191392B (en) * | 2021-04-07 | 2023-01-24 | 山东师范大学 | Breast cancer image information bottleneck multi-task classification and segmentation method and system |
| CN113379687A (en) * | 2021-05-28 | 2021-09-10 | 上海联影智能医疗科技有限公司 | Network training method, image detection method, and medium |
| CN113487656B (en) * | 2021-07-26 | 2022-11-22 | 推想医疗科技股份有限公司 | Image registration method and device, training method and device, control method and device |
| CN116071375B (en) * | 2023-03-10 | 2023-09-26 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Image segmentation method and device, storage medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8885893B1 (en) * | 2011-11-22 | 2014-11-11 | The United States Of America As Represented By The Secretary Of The Navy | System and method for adaptive face recognition |
| CN108304819A (en) * | 2018-02-12 | 2018-07-20 | 北京易真学思教育科技有限公司 | Gesture recognition system and method, storage medium |
| CN109660719A (en) * | 2018-12-11 | 2019-04-19 | 维沃移动通信有限公司 | A kind of information cuing method and mobile terminal |
-
2019
- 2019-12-23 CN CN201911338271.7A patent/CN111160367B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8885893B1 (en) * | 2011-11-22 | 2014-11-11 | The United States Of America As Represented By The Secretary Of The Navy | System and method for adaptive face recognition |
| CN108304819A (en) * | 2018-02-12 | 2018-07-20 | 北京易真学思教育科技有限公司 | Gesture recognition system and method, storage medium |
| CN109660719A (en) * | 2018-12-11 | 2019-04-19 | 维沃移动通信有限公司 | A kind of information cuing method and mobile terminal |
Non-Patent Citations (3)
| Title |
|---|
| Alexander Binder等."Enhanced representation and multi-task learning for image annotation".《Computer Vision and Image Understanding》.2013,第117卷(第5期),全文. * |
| Zhanpeng Zhang等."Facial Landmark Detection by Deep Multi-task Learning".《ECCV 2014: Computer Vision》.2014,第94-108页. * |
| 肖锋等."全卷积语义分割与物体检测网络".《中国图象图形学报》.2019,第24卷(第03期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111160367A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111160367B (en) | Image classification method, apparatus, computer device, and readable storage medium | |
| CN110766730B (en) | Image registration and follow-up evaluation method, storage medium and computer equipment | |
| US8958614B2 (en) | Image-based detection using hierarchical learning | |
| CN111080584B (en) | Quality control method for medical image, computer device and readable storage medium | |
| CN112885453A (en) | Method and system for identifying pathological changes in subsequent medical images | |
| CN111325714B (en) | Method for processing region of interest, computer device and readable storage medium | |
| US11915822B2 (en) | Medical image reading assistant apparatus and method for adjusting threshold of diagnostic assistant information based on follow-up examination | |
| CN110600107B (en) | Method for screening medical images, computer device and readable storage medium | |
| CN111311655B (en) | Multi-mode image registration method, device, electronic equipment and storage medium | |
| US20220036575A1 (en) | Method for measuring volume of organ by using artificial neural network, and apparatus therefor | |
| CN111080583B (en) | Medical image detection method, computer device, and readable storage medium | |
| EP3722996A2 (en) | Systems and methods for processing 3d anatomical volumes based on localization of 2d slices thereof | |
| CN112102235B (en) | Human body part recognition method, computer device, and storage medium | |
| CN111128348B (en) | Medical image processing method, medical image processing device, storage medium and computer equipment | |
| CN113935943A (en) | Method, device, computer equipment and storage medium for intracranial aneurysm identification detection | |
| CN111223158B (en) | Artifact correction method for heart coronary image and readable storage medium | |
| CN113808125A (en) | Medical image processing method, focus type identification method and related product | |
| CN111091539B (en) | Network model training and medical image processing methods, devices, mediums and equipment | |
| CN110570425A (en) | Lung nodule analysis method and device based on deep reinforcement learning algorithm | |
| CN113724185B (en) | Model processing method, device and storage medium for image classification | |
| CN114266896A (en) | Image labeling method, model training method and device, electronic equipment and medium | |
| CN110992312B (en) | Medical image processing method, medical image processing device, storage medium and computer equipment | |
| CN113160199A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN112766314A (en) | Anatomical structure recognition method, electronic device, and storage medium | |
| WO2023223060A1 (en) | Method for training a system adapted for aiding evaluation of a medical image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |