CN111160367A - Image classification method and device, computer equipment and readable storage medium - Google Patents

Image classification method and device, computer equipment and readable storage medium Download PDF

Info

Publication number
CN111160367A
CN111160367A CN201911338271.7A CN201911338271A CN111160367A CN 111160367 A CN111160367 A CN 111160367A CN 201911338271 A CN201911338271 A CN 201911338271A CN 111160367 A CN111160367 A CN 111160367A
Authority
CN
China
Prior art keywords
image
image classification
result
medical image
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911338271.7A
Other languages
Chinese (zh)
Other versions
CN111160367B (en
Inventor
詹恒泽
梁凯轶
周慧
郑介志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai United Imaging Intelligent Healthcare Co Ltd
Original Assignee
Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai United Imaging Intelligent Healthcare Co Ltd filed Critical Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority to CN201911338271.7A priority Critical patent/CN111160367B/en
Publication of CN111160367A publication Critical patent/CN111160367A/en
Application granted granted Critical
Publication of CN111160367B publication Critical patent/CN111160367B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/29Graphical models, e.g. Bayesian networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03Recognition of patterns in medical or anatomical images
    • G06V2201/031Recognition of patterns in medical or anatomical images of internal organs

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

The application relates to an image classification method, an image classification device, a computer device and a readable storage medium. The method comprises the following steps: acquiring a medical image to be classified; inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels; and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part. In the method, the computer equipment adopts a multitask network model to determine the shot part in the medical image, so that the accuracy of the result of the target shot part can be improved, and the accuracy of the subsequent focus identification process can be further improved; and the confirmation process is not needed by the radiologist, and the efficiency of the focus identification process is improved.

Description

Image classification method and device, computer equipment and readable storage medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image classification method, an image classification device, a computer device, and a readable storage medium.
Background
X-ray films (X-Rays) play an important role in early detection of lung diseases, heart diseases, abdominal diseases, fracture and the like due to low price and good imaging effect. Generally, after a scanning technician uses medical equipment to scan a patient, a medical image is submitted to a corresponding radiologist to read, and the radiologist identifies the lesion features in the medical image according to the experience of the radiologist and gives a lesion identification result.
Generally, when a scan technician scans a certain part of a patient, the scanned part label (for example, the current medical image is an abdominal image) is entered, and then a radiologist identifies the lesion in the medical image according to the part label.
However, in the conventional technique, the part label may be incorrectly recorded due to the error of the scanner technician, which requires the radiologist to determine the scanned part first and then perform the lesion recognition, and the efficiency and accuracy of the lesion recognition process are low.
Disclosure of Invention
Based on this, it is necessary to provide an image classification method, an apparatus, a computer device and a readable storage medium for solving the problem in the conventional art that the efficiency and accuracy of the lesion identification process are low.
In a first aspect, an embodiment of the present application provides an image classification method, including:
acquiring a medical image to be classified;
inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the training mode of the multitask network model comprises the following steps:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
and training the initial multitask network according to the first loss and the second loss to obtain a multitask network model.
In one embodiment, inputting a medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image, includes:
performing feature extraction on the medical image by adopting a first convolution layer in the multitask network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points;
and performing feature classification on the feature map by adopting a pooling layer and a full-link layer in the multitask network model to obtain an image classification result.
In one embodiment, when the segmentation result of the key point and the image classification result satisfy a preset condition, determining the shooting part represented by the image classification result as the target shooting part includes:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the corresponding relation between the preset number of key points and the image category;
and if so, determining the shooting part represented by the image classification result as the target shooting part.
In one embodiment, the target photographic part includes at least one of a part code, a part name, and a part orientation, and the part orientation includes a positive position or a lateral position.
In one embodiment, after determining that the capturing location characterized by the image classification result is the target capturing location, the method further includes:
acquiring a shooting label of the medical image, wherein the shooting label is shooting position data input by a user when the medical image is shot;
and if the shooting label is not consistent with the target shooting part, updating the shooting label to the target shooting part.
In one embodiment, after determining that the capturing location characterized by the image classification result is the target capturing location, the method further includes:
determining a focus detection algorithm corresponding to a target shooting part according to the target shooting part of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
In a second aspect, an embodiment of the present application provides an image classification apparatus, including:
the acquisition module is used for acquiring medical images to be classified;
the processing module is used for inputting the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and the determining module is used for determining the shooting part represented by the image classification result as the target shooting part when the segmentation result of the key point and the image classification result meet the preset conditions.
In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:
acquiring a medical image to be classified;
inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:
acquiring a medical image to be classified;
inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part.
The image classification method, the image classification device, the computer equipment and the readable storage medium can acquire the medical image to be classified; inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels; and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part. In the method, the computer equipment adopts a multitask network model to determine the shot part in the medical image, so that the accuracy of the result of the target shot part can be improved, and the accuracy of the subsequent focus identification process can be further improved; and the confirmation process is not needed by the radiologist, and the efficiency of the focus identification process is improved.
Drawings
FIG. 1 is a flowchart illustrating an image classification method according to an embodiment;
FIG. 1a is a diagram illustrating a multitasking network model according to an embodiment;
FIG. 1b is a diagram illustrating the correspondence between the number of keypoints and the image categories, according to an embodiment;
FIG. 1c is a schematic diagram of a target capture area and corresponding medical image provided in one embodiment;
fig. 2 is a schematic flowchart of an image classification method according to another embodiment;
FIG. 3 is a flowchart illustrating an image classification method according to another embodiment;
FIG. 4 is a flowchart illustrating an image classification method according to another embodiment;
FIG. 5 is a schematic structural diagram of an image classification apparatus according to an embodiment;
fig. 6 is a schematic structural diagram of an image classification apparatus according to another embodiment;
fig. 7 is a schematic internal structural diagram of a computer device according to an embodiment.
Detailed Description
The image classification method provided by the embodiment of the application can be applied to the process of classifying the shot medical images so as to determine the shooting positions of the medical images. The medical image may be an X-ray film, or may be a Computed Tomography (CT) image, a Nuclear Magnetic Resonance Image (MRI), a Positron Emission Tomography (PET) image, or the like. In the conventional technology, a scan technician usually enters a part label of a medical image, and a radiologist identifies a lesion of the medical image according to the part label, but the part label is incorrectly entered by the scan technician, so that the radiologist needs to re-judge a scanned part and identify the lesion, and the efficiency and the accuracy of a lesion identification process are low. The embodiment of the application provides an image classification method, an image classification device, a computer device and a readable storage medium, and aims to solve the technical problems.
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application are further described in detail by the following embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It should be noted that the execution subject of the method embodiments described below may be an image classification apparatus, which may be implemented as part of or all of a computer device by software, hardware, or a combination of software and hardware. The following method embodiments take the execution subject as a computer device for example, where the computer device may be a terminal, may also be a server, may be a separate computing device, and may also be integrated on a medical imaging device, which is not limited in this embodiment.
Fig. 1 is a flowchart illustrating an image classification method according to an embodiment. The embodiment relates to a specific process of judging a medical image to be classified by computer equipment to obtain a target shooting part of the medical image. As shown in fig. 1, the method includes:
s101, medical images to be classified are obtained.
In particular, the medical image to be classified is a captured image of a part of a patient, such as an X-ray image, which may be acquired by a computer device from a post-processing workstation or a Picture Archiving and Communication System (PACS). Optionally, the computer device may acquire the medical images uploaded to the PACS system by the radio technologist in real time, or may acquire all the medical images in the period from the PACS system at fixed time intervals. Optionally, the computer device may further obtain Medical images to be classified from a Hospital Information management System (HIS), a Clinical Information System (CIS), a Radiology Information management System (RIS), an Electronic Medical Record (EMR), and a related Medical image cloud storage platform.
S102, inputting the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels.
Specifically, the computer device may input the acquired medical image to be classified into a preset multitask network model, and the multitask network model may simultaneously implement image segmentation and image classification processes to obtain a segmentation result and an image classification result of a key point in the medical image. Wherein, the key points in the medical image are set according to the characteristics of different parts of the patient, for example, the wrist joint image comprises 5 key points corresponding to 5 fingers; the image classification result may characterize the captured region. The multitask network model can be obtained by training according to a training sample with key point labels and classification labels, and the multitask network model obtained by training can accurately segment and classify medical images. Optionally, the segmentation result of the keypoints may include the positions and numbers of the keypoints and the segmented keypoint images.
Optionally, before inputting the medical image into the multitasking network model, the computer device may further perform Data enhancement (Data augmentation) on the medical image: randomly turning the medical image left and right horizontally, carrying out translation transformation in the horizontal direction and the vertical direction, randomly rotating, filling edges and changing the contrast; then, the medical image is subjected to normalization and standardization preprocessing operation to obtain a standardized image and input the standardized image into the multitask network model.
Optionally, the multi-task network model may be a multilayer convolutional neural network, a network model formed by combining a partition network and a classification network, or other deep learning networks, which is not limited in this embodiment. When the multitask network model is a multilayer convolutional neural network, the network structure of the multitask network can be as shown in fig. 1a, the first half of the network is provided with extraction features, the second half of the network is provided with two branches, the first branch is mainly used for carrying out key point detection through segmentation, the second branch is mainly used for classifying the image types, and finally, segmentation results of key points and image classification results are output.
S103, when the segmentation result of the key points and the image classification result meet preset conditions, determining the shooting part represented by the image classification result as a target shooting part.
Specifically, when the segmentation result and the image classification result of the key point satisfy a preset condition, the computer device may determine that the captured portion represented by the image classification result is a target captured portion of the medical image. Optionally, the preset condition may be a correspondence between the number of key points and the image category, and the correspondence may be as shown in fig. 1b, for example, the wrist joint positive position image corresponds to 5 key points, and the chest side position corresponds to 10 key points, and only when the obtained key point segmentation result and the image classification result satisfy the correspondence, the target shooting position may be determined. Optionally, the preset condition may also be a correspondence between the key point positions and the image categories, for example, the 5 key point positions corresponding to the wrist joint positioning image are the finger joints of 5 fingers, and only when the obtained key point segmentation result is the finger joint positions of 5 fingers and the image classification result is the wrist joint positioning image, the target shooting position may be determined to be the wrist joint positioning.
Optionally, the target shooting part includes at least one of a part code, a part name, and a part orientation, where the part codes of different shooting parts are different, such as a wrist joint 0, a chest 1, an abdomen 2, a head 3, and the like, and the part orientation includes a normal position or a lateral position. The target capturing part and the corresponding medical image can be seen in fig. 1 c.
Furthermore, after the computer equipment determines the target shooting part, the target shooting part can be displayed to a radiologist, so that the radiologist can recognize the focus according to the target shooting part, and the focus recognition progress can be accelerated.
In the image classification method provided by this embodiment, the computer device inputs the acquired medical image to be classified into the multitask network model to obtain the segmentation result of the key points in the medical image and the image classification result, and when the segmentation result of the key points and the image classification result satisfy the preset condition, the shooting part represented by the image classification result can be determined as the target shooting part. In the method, the computer equipment adopts a multitask network model to determine the shot part in the medical image, so that the accuracy of the result of the target shot part can be improved, and the accuracy of the subsequent focus identification process can be further improved; and the confirmation process is not needed by the radiologist, and the efficiency of the focus identification process is improved.
In some embodiments, the training samples include a plurality of sample images and labels corresponding to each sample image, where the labels include a keypoint label and a classification label; as shown in fig. 2, the training method of the multitask network model includes:
s201, inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result.
Specifically, the initial multi-task network may be a newly built network, and the computer device inputs the sample image into the initial multi-task network to obtain an initial keypoint segmentation result and an initial image classification result. Since the accuracy of the initial multitask network is not high yet, the output result is the initial result. Optionally, the initial keypoint segmentation result may also include the position and number of the initial keypoints and the segmented keypoint image, and the initial image classification result is a captured portion in the sample image. The initial multitask network can also be divided into two parts, wherein the first half part of the network is used for extracting features, the second half part of the network is used for two branches, the first branch is mainly used for carrying out key point detection through segmentation, and the second branch is mainly used for classifying the categories of the images.
Optionally, before inputting the sample image into the initial multitask network, the computer device may also perform Data enhancement (Data augmentation) on the sample image: carrying out random left-right horizontal overturning, translation transformation in the horizontal direction and the vertical direction, random rotation, edge filling and contrast change on the sample image; and then, carrying out normalization and normalization preprocessing operation on the sample image to obtain a normalized image and inputting the normalized image into the initial multitask network.
S202, calculating a first loss between the initial key point segmentation result and the key point label and a second loss between the initial image classification result and the classification label.
Specifically, each sample image can be labeled in advance by an experienced doctor, that is, the positions, the numbers and the key point images of the labeled key points, and the categories of the sample images are used as the key point labels and the classification labels. The computer device then calculates a first loss between the obtained initial keypoint segmentation result and the keypoint labels, and a second loss between the initial image classification result and the classification labels.
Optionally, the computer device may use a cross entropy loss function to calculate the first loss and the second loss, or may use another type of loss function to calculate the loss, which is not limited in this embodiment.
S203, training the initial multitask network according to the first loss and the second loss to obtain a multitask network model.
In particular, the computer device may train the initial multitasking network according to the first loss and the second loss, i.e., adjust network parameters of the initial multitasking network according to the first loss and the second loss. Optionally, the computer device may add, sum, or weight, sum, or average the first loss and the second loss to obtain an overall loss to adjust the network parameter of the initial multitask network. And when the overall loss is less than or equal to a preset threshold value or convergence is reached, representing that the initial multi-task network training is completed, and obtaining the multi-task network model.
Optionally, in the training sample, one part may be selected as a training set, and the other part may be selected as a test set, and after the training set completes the initial multi-task network training, the computer device may further use the test set to test the trained network, so as to further ensure the accuracy of the multi-task network model.
In the image classification method provided by this embodiment, the computer device trains the initial multi-task network by using the training samples, and thus iterative training is performed, so that a multi-task network model with higher precision can be obtained, and the accuracy of the obtained key point segmentation result and the image classification result can be improved, thereby improving the accuracy of the lesion identification process.
As shown in fig. 1a, if the first half of the multitask network model is feature extraction and the second half is key point detection and classification, the step S102 may include: performing feature extraction on the medical image by adopting a first convolution layer in the multitask network model to obtain a feature map of the medical image; performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points; and performing feature classification on the feature map by adopting a pooling layer and a full-link layer in the multitask network model to obtain an image classification result.
The method comprises the steps that a feature map of a medical image can be obtained by performing convolution on a plurality of convolution layers of a multitask network model, and then key point feature detection is performed on the feature map through another plurality of convolution layers to obtain a segmentation result of key points, namely a key point image and a background image are distinguished; and mapping and classifying the features in the feature map through the pooling layer and the full-connection layer to obtain an image classification result.
Optionally, in some embodiments, the S103 may include: determining the number of key points according to the segmentation result of the key points; judging whether the number of key points and the image classification result meet the corresponding relation between the preset number of key points and the image category; and if so, determining the shooting part represented by the image classification result as the target shooting part.
When the segmentation result of the key point only includes the position of the key point or the segmented image, the computer device may determine the number of the key points according to the number of the positions or the number of the segmented images, and then may determine whether the number of the key points determined by the multitask network model and the image classification result satisfy the correspondence according to a preset correspondence between the number of the key points and the image category, such as the correspondence shown in fig. 1 b. And if so, taking the shooting part represented by the image classification result as a target shooting part.
Optionally, in some embodiments, as shown in fig. 3, the method further includes:
s301, acquiring a shooting label of the medical image, wherein the shooting label is shooting position data recorded by a user when the medical image is shot.
And S302, if the shooting label is not consistent with the target shooting part, updating the shooting label to the target shooting part.
Specifically, the computer device may acquire a shooting tag carried by the medical image, which is recorded by the scanning technician when the medical image is captured, such as a chest tag, a head tag, an abdomen tag, and the like. The computer device then determines whether or not the imaging tag entered by the scanner technician matches the target imaging region, and if not, updates the imaging tag to the target imaging tag. For example, if the imaging tag entered by the scanning technician is a chest tag and the target imaging region obtained is an abdominal true position, which are not coincident with each other, and it is considered that the entry by the scanning technician is incorrect, the computer device updates the chest tag to an abdominal tag. Therefore, timely correction can be performed when errors occur in related data of the medical image, so that a data base is made for subsequent other data analysis, and other data analysis processes are not influenced.
Optionally, in some embodiments, as shown in fig. 4, after the target capturing region is determined, the method further includes:
s401, according to the target shooting part of the medical image, determining a focus detection algorithm corresponding to the target shooting part.
S402, detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
Specifically, the computer device may determine a lesion detection algorithm corresponding to the target shooting location according to the obtained target shooting location, and the lesion detection algorithm may be stored in an algorithm library of the computer device. If the target shooting part is a lung positive film, a pulmonary nodule detection algorithm, an emphysema detection algorithm and the like can be called to carry out focus detection on the target shooting part; when the target shooting part is a head positive film, a cerebral hemorrhage detection algorithm, a cerebroma detection algorithm and the like can be called to carry out focus detection on the target shooting part; when the target shooting part is the lateral position of the knee joint, a fracture detection algorithm can be called to detect the target shooting part, and the like. Optionally, the lesion detection algorithm may be a neural network algorithm, or may be other types of algorithms, which is not limited in this embodiment.
In the image classification method of the embodiment, after the shooting position of the medical image is judged by using the computer equipment, a corresponding focus detection algorithm can be automatically called to detect the medical image, so that a focus detection result is obtained. The full automation of the focus identification process can be realized, no human participation is needed, and the efficiency and the accuracy of the focus identification process can be further improved.
It should be understood that although the various steps in the flowcharts of fig. 1-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
Fig. 5 is a schematic structural diagram of an image classification apparatus according to an embodiment. As shown in fig. 5, the apparatus includes: an acquisition module 11, a processing module 12 and a determination module 13.
Specifically, the acquiring module 11 is configured to acquire a medical image to be classified.
The processing module 12 is configured to input the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of a key point in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels.
And the determining module 13 is configured to determine, when the segmentation result of the key point and the image classification result meet a preset condition, that the shooting part represented by the image classification result is the target shooting part.
The image classification apparatus provided in this embodiment may implement the method embodiments described above, and the implementation principle and the technical effect are similar, which are not described herein again.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; as shown in fig. 6, the apparatus further comprises a training module 14.
Specifically, the training module 14 is configured to input the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result; calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label; and training the initial multitask network according to the first loss and the second loss to obtain a multitask network model.
The image classification apparatus provided in this embodiment may implement the method embodiments described above, and the implementation principle and the technical effect are similar, which are not described herein again.
In one embodiment, the processing module 12 is specifically configured to perform feature extraction on a medical image by using a first convolution layer in a multitask network model to obtain a feature map of the medical image; performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points; and performing feature classification on the feature map by adopting a pooling layer and a full-link layer in the multitask network model to obtain an image classification result.
In one embodiment, the determining module 13 is specifically configured to determine the number of key points according to a segmentation result of the key points; judging whether the number of key points and the image classification result meet the corresponding relation between the preset number of key points and the image category; and if so, determining the shooting part represented by the image classification result as the target shooting part.
In one embodiment, the target photographic part includes at least one of a part code, a part name, and a part orientation, and the part orientation includes a positive position or a lateral position.
In one embodiment, the apparatus further comprises an update module; the acquisition module 11 is further configured to acquire a shooting tag of the medical image, where the shooting tag is shooting location data entered by a user when the medical image is shot; and the updating module is used for updating the shooting label to the target shooting part if the shooting label is inconsistent with the target shooting part.
In one embodiment, the apparatus further includes a detection module, configured to determine a lesion detection algorithm corresponding to a target shooting location of the medical image; and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
The image classification apparatus provided in this embodiment may implement the method embodiments described above, and the implementation principle and the technical effect are similar, which are not described herein again.
For the specific definition of the image classification device, reference may be made to the above definition of the image classification method, which is not described herein again. The modules in the image classification device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image classification method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring a medical image to be classified;
inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part.
The implementation principle and technical effect of the computer device provided in this embodiment are similar to those of the method embodiments described above, and are not described herein again.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the processor, when executing the computer program, further performs the steps of:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
and training the initial multitask network according to the first loss and the second loss to obtain a multitask network model.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
performing feature extraction on the medical image by adopting a first convolution layer in the multitask network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points;
and performing feature classification on the feature map by adopting a pooling layer and a full-link layer in the multitask network model to obtain an image classification result.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the corresponding relation between the preset number of key points and the image category;
and if so, determining the shooting part represented by the image classification result as the target shooting part.
In one embodiment, the target photographic part includes at least one of a part code, a part name, and a part orientation, the part orientation including a positive or lateral position.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring a shooting label of the medical image, wherein the shooting label is shooting position data input by a user when the medical image is shot;
and if the shooting label is not consistent with the target shooting part, updating the shooting label to the target shooting part.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
determining a focus detection algorithm corresponding to a target shooting part according to the target shooting part of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a medical image to be classified;
inputting the medical image into a preset multi-task network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet the preset condition, determining the shooting part represented by the image classification result as the target shooting part.
The implementation principle and technical effect of the computer-readable storage medium provided by this embodiment are similar to those of the above-described method embodiment, and are not described herein again.
In one embodiment, the training sample comprises a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise key point labels and classification labels; the computer program when executed by the processor further realizes the steps of:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint label and a second loss between the initial image classification result and the classification label;
and training the initial multitask network according to the first loss and the second loss to obtain a multitask network model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing feature extraction on the medical image by adopting a first convolution layer in the multitask network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points;
and performing feature classification on the feature map by adopting a pooling layer and a full-link layer in the multitask network model to obtain an image classification result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining the number of key points according to the segmentation result of the key points;
judging whether the number of key points and the image classification result meet the corresponding relation between the preset number of key points and the image category;
and if so, determining the shooting part represented by the image classification result as the target shooting part.
In one embodiment, the target photographic part includes at least one of a part code, a part name, and a part orientation, the part orientation including a positive or lateral position.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a shooting label of the medical image, wherein the shooting label is shooting position data input by a user when the medical image is shot;
and if the shooting label is not consistent with the target shooting part, updating the shooting label to the target shooting part.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining a focus detection algorithm corresponding to a target shooting part according to the target shooting part of the medical image;
and detecting the medical image according to a focus detection algorithm to obtain a focus detection result.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. An image classification method, comprising:
acquiring a medical image to be classified;
inputting the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and when the segmentation result of the key point and the image classification result meet a preset condition, determining the shooting part represented by the image classification result as a target shooting part.
2. The method of claim 1, wherein the training samples comprise a plurality of sample images and labels corresponding to each sample image, the labels comprising keypoint labels and classification labels; the training mode of the multitask network model comprises the following steps:
inputting the sample image into an initial multi-task network to obtain an initial key point segmentation result and an initial image classification result;
calculating a first loss between the initial keypoint segmentation result and the keypoint labels and a second loss between the initial image classification result and the classification labels;
and training the initial multitask network according to the first loss and the second loss to obtain the multitask network model.
3. The method according to claim 1 or 2, wherein the inputting the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image comprises:
performing feature extraction on the medical image by adopting a first convolution layer in the multitask network model to obtain a feature map of the medical image;
performing key point feature detection on the feature map by adopting a second convolution layer in the multitask network model to obtain a segmentation result of the key points;
and carrying out feature classification on the feature graph by adopting a pooling layer and a full-link layer in the multitask network model to obtain the image classification result.
4. The method according to claim 1 or 2, wherein when the segmentation result of the key point and the image classification result satisfy a preset condition, determining that the shot part represented by the image classification result is a target shot part comprises:
determining the number of the key points according to the segmentation result of the key points;
judging whether the number of the key points and the image classification result meet the corresponding relation between the preset number of the key points and the image category;
and if so, determining the shooting part represented by the image classification result as a target shooting part.
5. The method of claim 4, wherein the target photographic part comprises at least one of a part code, a part name, and a part orientation, the part orientation comprising a positive or lateral position.
6. The method of claim 1, wherein after determining that the capture region characterized by the image classification result is a target capture region, the method further comprises:
acquiring a shooting label of the medical image, wherein the shooting label is shooting position data input by a user when the medical image is shot;
and if the shooting label is not consistent with the target shooting part, updating the shooting label to the target shooting part.
7. The method of claim 1, wherein after determining that the capture region characterized by the image classification result is a target capture region, the method further comprises:
determining a focus detection algorithm corresponding to a target shooting part according to the target shooting part of the medical image;
and detecting the medical image according to the focus detection algorithm to obtain a focus detection result.
8. An image classification apparatus, comprising:
the acquisition module is used for acquiring medical images to be classified;
the processing module is used for inputting the medical image into a preset multitask network model to obtain a segmentation result and an image classification result of key points in the medical image; the multi-task network model is obtained by training according to a training sample with key point labels and classification labels;
and the determining module is used for determining the shooting part represented by the image classification result as a target shooting part when the segmentation result of the key point and the image classification result meet preset conditions.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
CN201911338271.7A 2019-12-23 2019-12-23 Image classification method, apparatus, computer device, and readable storage medium Active CN111160367B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911338271.7A CN111160367B (en) 2019-12-23 2019-12-23 Image classification method, apparatus, computer device, and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911338271.7A CN111160367B (en) 2019-12-23 2019-12-23 Image classification method, apparatus, computer device, and readable storage medium

Publications (2)

Publication Number Publication Date
CN111160367A true CN111160367A (en) 2020-05-15
CN111160367B CN111160367B (en) 2024-03-01

Family

ID=70557819

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911338271.7A Active CN111160367B (en) 2019-12-23 2019-12-23 Image classification method, apparatus, computer device, and readable storage medium

Country Status (1)

Country Link
CN (1) CN111160367B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111640093A (en) * 2020-05-20 2020-09-08 上海联影智能医疗科技有限公司 Quality control method for medical image and computer-readable storage medium
CN111652300A (en) * 2020-05-27 2020-09-11 联影智能医疗科技(北京)有限公司 Spine curvature classification method, computer device and storage medium
CN111696083A (en) * 2020-05-20 2020-09-22 平安科技(深圳)有限公司 Image processing method and device, electronic equipment and storage medium
CN111932547A (en) * 2020-09-24 2020-11-13 平安科技(深圳)有限公司 Method and device for segmenting target object in image, electronic device and storage medium
CN112766314A (en) * 2020-12-31 2021-05-07 上海联影智能医疗科技有限公司 Anatomical structure recognition method, electronic device, and storage medium
CN112802036A (en) * 2021-03-16 2021-05-14 上海联影医疗科技股份有限公司 Method, system and device for segmenting target area of three-dimensional medical image
CN113140290A (en) * 2020-12-17 2021-07-20 慧影医疗科技(北京)有限公司 Model training method and device, focus identification method and electronic equipment
CN113191392A (en) * 2021-04-07 2021-07-30 山东师范大学 Breast cancer image information bottleneck multi-task classification and segmentation method and system
CN113379687A (en) * 2021-05-28 2021-09-10 上海联影智能医疗科技有限公司 Network training method, image detection method, and medium
CN113487656A (en) * 2021-07-26 2021-10-08 推想医疗科技股份有限公司 Image registration method and device, training method and device, control method and device
WO2022188547A1 (en) * 2021-03-09 2022-09-15 杭州朝厚信息科技有限公司 Method for detecting key point of x-ray cephalogram image
US11507776B2 (en) 2020-10-14 2022-11-22 Industrial Technology Research Institute Image recognition method for detection tasks based on single convolutional neural network and image recognition system thereof
CN116071375A (en) * 2023-03-10 2023-05-05 福建自贸试验区厦门片区Manteia数据科技有限公司 Image segmentation method and device, storage medium and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8885893B1 (en) * 2011-11-22 2014-11-11 The United States Of America As Represented By The Secretary Of The Navy System and method for adaptive face recognition
CN108304819A (en) * 2018-02-12 2018-07-20 北京易真学思教育科技有限公司 Gesture recognition system and method, storage medium
CN109660719A (en) * 2018-12-11 2019-04-19 维沃移动通信有限公司 A kind of information cuing method and mobile terminal

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8885893B1 (en) * 2011-11-22 2014-11-11 The United States Of America As Represented By The Secretary Of The Navy System and method for adaptive face recognition
CN108304819A (en) * 2018-02-12 2018-07-20 北京易真学思教育科技有限公司 Gesture recognition system and method, storage medium
CN109660719A (en) * 2018-12-11 2019-04-19 维沃移动通信有限公司 A kind of information cuing method and mobile terminal

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ALEXANDER BINDER等: ""Enhanced representation and multi-task learning for image annotation"" *
ZHANPENG ZHANG等: ""Facial Landmark Detection by Deep Multi-task Learning"" *
肖锋等: ""全卷积语义分割与物体检测网络"" *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111696083A (en) * 2020-05-20 2020-09-22 平安科技(深圳)有限公司 Image processing method and device, electronic equipment and storage medium
CN111640093A (en) * 2020-05-20 2020-09-08 上海联影智能医疗科技有限公司 Quality control method for medical image and computer-readable storage medium
CN111696083B (en) * 2020-05-20 2024-05-14 平安科技(深圳)有限公司 Image processing method and device, electronic equipment and storage medium
WO2021114638A1 (en) * 2020-05-20 2021-06-17 平安科技(深圳)有限公司 Image processing method, apparatus, electronic device, and storage medium
CN111652300A (en) * 2020-05-27 2020-09-11 联影智能医疗科技(北京)有限公司 Spine curvature classification method, computer device and storage medium
CN111932547A (en) * 2020-09-24 2020-11-13 平安科技(深圳)有限公司 Method and device for segmenting target object in image, electronic device and storage medium
CN111932547B (en) * 2020-09-24 2021-06-11 平安科技(深圳)有限公司 Method and device for segmenting target object in image, electronic device and storage medium
US11507776B2 (en) 2020-10-14 2022-11-22 Industrial Technology Research Institute Image recognition method for detection tasks based on single convolutional neural network and image recognition system thereof
TWI812888B (en) * 2020-10-14 2023-08-21 財團法人工業技術研究院 Image recognition method and image recognition system
CN113140290A (en) * 2020-12-17 2021-07-20 慧影医疗科技(北京)有限公司 Model training method and device, focus identification method and electronic equipment
CN112766314A (en) * 2020-12-31 2021-05-07 上海联影智能医疗科技有限公司 Anatomical structure recognition method, electronic device, and storage medium
CN112766314B (en) * 2020-12-31 2024-05-28 上海联影智能医疗科技有限公司 Anatomical structure recognition method, electronic device, and storage medium
WO2022188547A1 (en) * 2021-03-09 2022-09-15 杭州朝厚信息科技有限公司 Method for detecting key point of x-ray cephalogram image
CN112802036A (en) * 2021-03-16 2021-05-14 上海联影医疗科技股份有限公司 Method, system and device for segmenting target area of three-dimensional medical image
CN113191392A (en) * 2021-04-07 2021-07-30 山东师范大学 Breast cancer image information bottleneck multi-task classification and segmentation method and system
CN113379687A (en) * 2021-05-28 2021-09-10 上海联影智能医疗科技有限公司 Network training method, image detection method, and medium
CN113487656A (en) * 2021-07-26 2021-10-08 推想医疗科技股份有限公司 Image registration method and device, training method and device, control method and device
CN116071375A (en) * 2023-03-10 2023-05-05 福建自贸试验区厦门片区Manteia数据科技有限公司 Image segmentation method and device, storage medium and electronic equipment
CN116071375B (en) * 2023-03-10 2023-09-26 福建自贸试验区厦门片区Manteia数据科技有限公司 Image segmentation method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN111160367B (en) 2024-03-01

Similar Documents

Publication Publication Date Title
CN111160367B (en) Image classification method, apparatus, computer device, and readable storage medium
CN110766730B (en) Image registration and follow-up evaluation method, storage medium and computer equipment
WO2022141882A1 (en) Lesion recognition model construction apparatus and system based on historical pathological information
CN111161270A (en) Blood vessel segmentation method for medical image, computer device and readable storage medium
CN111080573B (en) Rib image detection method, computer device and storage medium
CN112885453A (en) Method and system for identifying pathological changes in subsequent medical images
CN110600107B (en) Method for screening medical images, computer device and readable storage medium
CN111311655B (en) Multi-mode image registration method, device, electronic equipment and storage medium
CN111325714B (en) Method for processing region of interest, computer device and readable storage medium
CN111080584A (en) Quality control method for medical image, computer device and readable storage medium
US11996198B2 (en) Determination of a growth rate of an object in 3D data sets using deep learning
EP3895600A1 (en) Method for measuring volume of organ by using artificial neural network, and apparatus therefor
CN114092475B (en) Focal length determining method, image labeling method, device and computer equipment
CN111223158B (en) Artifact correction method for heart coronary image and readable storage medium
CN111128345A (en) Medical image acquisition method, medical scanning device and computer storage medium
CN112102235B (en) Human body part recognition method, computer device, and storage medium
CN111128348B (en) Medical image processing method, medical image processing device, storage medium and computer equipment
CN113935943A (en) Method, device, computer equipment and storage medium for intracranial aneurysm identification detection
CN111091539B (en) Network model training and medical image processing methods, devices, mediums and equipment
CN116469545A (en) Method, device and equipment for carrying out auxiliary disease prediction by using medical image
CN113160199B (en) Image recognition method and device, computer equipment and storage medium
CN110570425A (en) Lung nodule analysis method and device based on deep reinforcement learning algorithm
CN113724185B (en) Model processing method, device and storage medium for image classification
CN114266896A (en) Image labeling method, model training method and device, electronic equipment and medium
CN110992312B (en) Medical image processing method, medical image processing device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant