CN114565582A

CN114565582A - Medical image classification and lesion area positioning method, system and storage medium

Info

Publication number: CN114565582A
Application number: CN202210198514.7A
Authority: CN
Inventors: 陈海欣; 杨雪松; 邓晓; 陈思
Original assignee: Foshan Map Reading Technology Co ltd
Current assignee: Foshan Map Reading Technology Co ltd
Priority date: 2022-03-01
Filing date: 2022-03-01
Publication date: 2022-05-31
Anticipated expiration: 2042-03-01
Also published as: CN114565582B

Abstract

The invention relates to the technical field of medical image identification and detection, in particular to a medical image classification and lesion area positioning method, a medical image classification and lesion area positioning system and a storage medium, wherein the method comprises the following steps of 1: receiving a medical image intercepted by medical imaging equipment in real time; step 2: detecting and classifying the medical images; and step 3: according to the detection classification result in the step 2, when the detection classification result comprises that the image scanning completion degree is 100%, calling a detection positioning algorithm of the corresponding type of image to detect the potential interested organ and the focus area to obtain detection information; and 4, step 4: according to the detection information, drawing and marking are carried out on the medical image, and a marked auxiliary image is obtained; and 5: and (5) sending the auxiliary image obtained in the step (4) back to the medical image equipment. The invention can be shared by a plurality of medical image devices, and the drawing and labeling of the medical image can be realized without changing the existing medical image devices or requiring the control software to provide an interface.

Description

Medical image classification and lesion area positioning method, system and storage medium

Technical Field

The invention relates to the technical field of medical image identification and detection, in particular to a medical image classification and lesion area positioning method, system and storage medium.

Background

In the process of scanning a patient by a medical imaging device, there are many cases in which a multi-step scanning is required, and the next scanning scheme (parameters, position, etc.) is usually determined based on the analysis of the current scanning result by an operator. If the axial position area and the scanning sequence parameter of the next tomography are judged on the basis of the CT scout image, the position, the slice angle and the scanning sequence parameter of the next local tomography are determined on the basis of the large-range magnetic resonance scanning, and the position and the acquisition parameter of the next local tomography are determined on the basis of the SPECT whole-body plain scanning.

The on-line analysis of the images during acquisition to determine the next scanning protocol is a very challenging and difficult task for the equipment operator. Firstly, the equipment operator is usually not a skilled physician and is inexperienced in image judgment; secondly, the operating software of the medical imaging device usually lacks tools for assisting image analysis; thirdly, a quick decision is needed in the scanning process, so that the bad experience of the patient caused by long-time delay is avoided.

At present, some medical imaging equipment manufacturers develop their own online image analysis modules, for example, organ detection is performed on CT scout images, so as to guide positioning of tomography. On one hand, the development of related functions belongs to the design change of the existing equipment (software), and needs strict process approval, so that the time of coming into the market is longer; on the other hand, only a subset of device manufacturers currently prefer or are implementing such functionality, while at the same time preferring to offer such functionality at a reasonable price to users, particularly those with longer-market devices.

Therefore, the number of products with the function is still less in clinical practice at present. Therefore, in some medical image scanning, due to inexperience or time-critical, the acquired images are inevitably incomplete, inaccurate or degraded, thereby affecting further diagnosis.

Disclosure of Invention

The method, the system and the computer readable storage medium for classifying medical images and positioning lesion areas are provided to solve the problems that the operation software of the existing medical imaging equipment provided in the background art lacks a general auxiliary image analysis tool and can be matched with experienced and skilled doctors to draw and mark images acquired by the medical imaging equipment.

In order to achieve the purpose, the invention adopts the following technical scheme:

a medical image classification and lesion region localization method comprises the following steps:

step 1: receiving a medical image intercepted by medical imaging equipment in real time;

step 2: detecting and classifying the medical images;

and step 3: according to the detection classification result in the step 2, when the detection classification result comprises that the image scanning completion degree is 100%, calling a detection positioning algorithm of the corresponding type of image to detect the potential interested organ and the focus area to obtain detection information;

and 4, step 4: according to the detection information, drawing and marking are carried out on the medical image, and a marked auxiliary image is obtained;

and 5: and (4) sending the auxiliary image feedback obtained in the step (4) to the medical imaging equipment.

Preferably, the medical image comprises a scout image, a scout image or a whole body image;

the detection classification is to detect whether a scanned medical image exists, and meanwhile, the completion degree of the medical image is evaluated according to a pre-designed completion degree classification standard, or the completion condition of the compared medical image is evaluated according to the analysis of the medical image change obtained from the same medical imaging equipment in two times.

Preferably, the algorithm of YOLO v5 is adopted in the step 2.

Preferably, the detection positioning algorithm comprises a RetinaNet network module and a CenterNet network module;

when the medical image is a positioning image or a scout image, calling a RetinaNet network module to detect potential interested organs and focus areas to obtain detection information;

and when the medical image is a whole body image, calling a RetinaNet network module to detect the potential interested organs and focus areas to obtain detection information.

A medical image classification and lesion area positioning system comprises a medical imaging device and an image analysis server, wherein the medical imaging device is in communication connection with the image analysis server;

the medical image equipment comprises a screenshot module and a sending module, wherein the screenshot module is used for intercepting a medical image of the medical image equipment in real time, and the sending module is used for sending the medical image to the image analysis server;

the image analysis server comprises a receiving module, a detection and classification module, a calling module, a sketching and labeling module and a feedback sending module;

the receiving module is used for receiving the medical image intercepted by the medical imaging equipment in real time;

the detection classification module is used for detecting and classifying the medical images and outputting detection classification results;

the calling module is used for calling a detection positioning algorithm of the corresponding type of image when the detection classification result comprises that the image scanning completion degree is% so as to detect the potential interested organ and the focus area and obtain the detection information;

the drawing and marking module is used for drawing and marking the medical image according to the detection information to obtain an auxiliary image after marking;

the feedback sending module is used for sending the auxiliary image to the medical image equipment in a feedback mode.

Preferably, the medical imaging device further comprises a timing module, and the timing module is used for driving the screenshot module in a timing manner so as to capture the medical image which is being scanned or is completed by scanning in a timing manner and in real time.

the detection classification is to detect whether a medical image is scanned or not, and meanwhile, the completion degree of the medical image is evaluated according to a pre-designed completion degree classification standard, or the completion condition of the medical image is evaluated according to the analysis of the change contrast of the medical image obtained from the same medical imaging equipment twice.

Preferably, the detection classification module 22 is deployed with the algorithm of YOLO v 5.

A storage medium, which is a computer-readable storage medium, having a medical image classification and lesion region localization program stored thereon, which when executed by a processor, implements the steps of one of the medical image classification and lesion region localization methods described above.

Compared with the prior art, the technical scheme has the following beneficial effects:

(1) the method comprises the following steps: firstly, receiving a medical image intercepted by medical imaging equipment in real time for detection classification analysis, calling a detection positioning algorithm of a corresponding type of image when a detection classification result comprises an image completion scanning evaluation degree of 100%, obtaining detection information through the operation of the detection positioning algorithm, performing drawing and marking on the medical image according to the detection information, and obtaining an auxiliary image after marking; meanwhile, according to the labeling of the organ on the fed-back auxiliary image, the instrument is more convenient to operate when an operator scans the specific organ part, the instrument is aligned to the center of the organ, incomplete scanning or unnecessary scanning beds are avoided due to the fact that the scanning beds are not aligned, and the operator is assisted to better implement next scanning diagnosis.

(2) The system of the invention adopts a distributed architecture and consists of two parts, namely medical image equipment (client) and an image analysis server (server), as shown in the figure. The functions and complexity of client software running on an operating computer of the medical imaging equipment are simplified: the medical image of the medical image equipment is intercepted in real time and uploaded to a service software end of an image analysis server, after the detection classification step and the detection positioning step of the image analysis server, client software of a medical image equipment operation computer can obtain an auxiliary image with a hook and a label, which is fed back and sent by the service software end of the image analysis server, and the auxiliary image is clicked and displayed by an operator, so that the influence and the interference on the medical image equipment operation computer are avoided.

Drawings

FIG. 1 is a flow chart of a medical image classification and lesion region localization method according to the present invention;

FIG. 2 is a schematic diagram of the medical image classification and lesion region localization system of the present invention;

FIG. 3 is a schematic diagram of the network framework of the YOLO v5 algorithm of the present invention;

FIG. 4 is a schematic diagram of the detection of CT scout image in medical imaging equipment according to the present invention;

FIG. 5 is a schematic diagram of the detection of a SPECT whole-body planar image in a medical imaging device according to the present invention;

fig. 6 is a schematic structural diagram of a RetinaNet network module according to the present invention;

FIG. 7 is a schematic diagram illustrating the method for detecting delineation and labeling of different regions in a CT scout image by using a RetinaNet network module in an image analysis server according to the present invention;

figure 8 is a schematic diagram of the architecture of the centrnet network module of the present invention;

FIG. 9 is a drawing and labeling diagram of SPECT whole-body planar image height capturing organs and lesion areas detected by invoking a CenterNet network module in an image analysis server according to the present invention.

In the drawings: the medical image processing system comprises medical image equipment 1, an image analysis server 2, a screenshot module 11, a sending module 12, a timing module 13, a receiving module 21, a detection and classification module 22, a calling module 23, a sketching and labeling module 24 and a feedback sending module 25.

Detailed Description

The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.

In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.

In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.

step 2: detecting and classifying the medical images;

and 5: and (5) sending the auxiliary image obtained in the step (4) back to the medical image equipment.

The method can be shared by a plurality of medical imaging devices, and can realize the delineation and marking of medical images without changing the existing medical imaging devices or requiring the existing medical imaging device control software to provide an interface, thereby solving the problem that the existing users using the medical imaging devices need to match experienced intensive medical doctors to delineate and mark the images acquired by the medical imaging devices, and simultaneously solving the problem that the same user needs to match a plurality of intensive medical doctors with different experiences because the types of the medical images are many and the intensive medical doctors are not familiar with each type of medical images.

Specifically, the flow of the method provided by the invention is shown in fig. 1, firstly, a medical image intercepted by medical imaging equipment in real time is received for detection classification analysis, when the detection classification result comprises an image completeness scanning evaluation degree of 100%, a detection positioning algorithm of a corresponding type image is called, detection information is obtained through the operation of the detection positioning algorithm, drawing and marking are carried out on the medical image according to the detection information, and a marked auxiliary image is obtained; meanwhile, according to the labeling of the organ on the fed-back auxiliary image, the instrument is more convenient to operate when an operator scans the specific organ part, the instrument is aligned to the center of the organ, incomplete scanning or unnecessary scanning beds are avoided due to the fact that the scanning beds are not aligned, and the operator is assisted to better implement next scanning diagnosis.

By way of further illustration, the medical image comprises a scout image, or a whole body image;

the detection classification is to detect whether a medical image is scanned or not, and meanwhile, the completion degree of the medical image is evaluated according to a pre-designed completion degree classification standard, or the completion condition of the medical image obtained after comparison is evaluated according to the analysis of medical image changes obtained from the same medical imaging equipment in two times.

The scout view refers to a two-dimensional X-ray planar image obtained by planar scanning of an imaging target before tomography by CT, as shown in fig. 4.

The scout image is an image obtained by rapidly scanning a certain region before a regular scan is performed by CT, Magnetic Resonance Imaging (MRI), SPECT (single photon emission tomography) or PET (positron emission tomography), and the image is usually lower in signal-to-noise ratio and resolution than a regular scan image and is used for obtaining preliminary information so that an operator can determine the part and parameters of the next scan.

The whole-body image mainly refers to an imaging mode of performing whole-body scanning in clinical application of nuclear medicine aiming at tumors and the like so as to find and locate primary tumors and metastatic foci. Including both whole-body plain imaging and whole-body tomographic imaging, as shown in fig. 5. In addition to whole-body imaging, there is a possibility that additional tomographic imaging or delayed imaging may be performed on a certain region as necessary for diagnosis.

To explain further, the step 2 adopts the algorithm of YOLO v 5. The output of the YOLO v5 algorithm at the detection section typically contains the starting coordinates and length and width of the top left corner of the detection box, the confidence that the detection box contains a detected object, and the confidence that the detection box is in each classification category.

In the currently detected medical image categories, the scout image and the whole body image belong to mutually exclusive categories, categories with different degrees of completion also belong to mutually exclusive categories, and when the image category and the categories with different degrees of completion exist simultaneously, if the image category and the categories with different degrees of completion are combined, more groups (image category × degree of completion category) can be generated, for example, 50% degree of completion of the scout image, 75% degree of completion of the scout image and the like, generally more data are needed to train the YOLO v5 algorithm. Because each image category comprises different completion degree categories, the mode of respectively performing Softmax functions on different groups is adopted at present, artificial shielding with different degrees is randomly performed on training images in the network training process, and completion degree classification is marked; the neck part is used for fusing and processing the feature layers of different sizes/levels of the backbone network to enhance the expression capability of the network; the header is used to predict the location of the detection box and the category information.

The effect of the Softmax function is to make the sum of the probabilities of the class types 1. On one hand, the types with the highest probability in the respective groups can be obtained by respectively using Softmax functions for different mutually exclusive groups, on the other hand, data of multiple groups of types can be trained simultaneously, and the training time is shortened, if the training data does not necessarily contain the 50% completeness of the whole body image, the classification of the 50% completeness of the whole body image can be predicted according to the data of the 50% completeness of other image types of the training data.

Although other classification algorithms exist at present, the YOLO v5 algorithm in the method is the optimal solution in the scene, because the YOLO v5 algorithm has relatively high detection speed under certain detection accuracy, and is more convenient for an operator to grasp the progress of image completion degree in real time. And in the classification step, errors are not easy to occur due to large difference between the detected image types.

Further, the YOLO v5 algorithm also uses mosaic picture enhancement training and adaptive anchor frame and other technologies, the main network is a cross-stage local network, and detection of targets with different scales is realized through a Neck-path aggregation network and a Head-YOLO general detection layer so as to adapt to scale differences of monitoring images in different medical image equipment operation software, as shown in fig. 3.

For further illustration, the detection and location algorithm includes a RetinaNet network module and a CenterNet network module;

Example 1

When the medical image is determined to be a CT scout image by detection classification, a RetinaNet network model is called to detect different axial anatomical regions of the CT scout image to assist in guiding positioning, as shown in fig. 6. The network model adopts the resnet50 as a basic network to extract features, then, FPN (feature space pyramid) is used for multi-dimension prediction, a feature map with three dimensions of large, medium and small is output together, each output is two paths for classification and regression of a target frame, and 9 anchor frames are adopted during output and comprise combinations of 3 dimensions and 3 length-width ratios, so that the model is suitable for different detection frame dimensions in medical images. The head parts (parts c and d in fig. 4) of different layers of the whole structure share parameters, which is beneficial to reducing the parameter number of the model, so that the operation is simpler and more efficient, but because the categories of the tasks are inconsistent, the parameters between the classification branches and the regression branches are not shared, the bias of the last stage convolution of the classification branches is initialized, and the detection result is shown in fig. 7.

Example 2

When the medical image is determined to be the SPECT whole-body plain image by detection classification, a CenterNet network module is called to carry out high-uptake organ and lesion detection on the SPECT whole-body plain image, as shown in FIG. 8, a backbone part network adopts Resnet50, the output size is (b,16 and 2048), a discarding layer is added, and the disconnection is carried out at random by 50%. The feature map scale was changed to (128 ) by three upsampling, consistent with a thermodynamic map (heatmap) obtained by gaussian radius. The input data ensures that the minimum edge is not less than 512 through affine transformation, and the deformation of the image caused by the affine transformation is prevented from being too large. Adding in the data enhancement part: visual changes such as random contrast, brightness, sharpness changes, histogram equalization, etc., as well as conventional rotation, shearing, translation, scaling, etc. Loss function: the heatmap uses focalloss, the width and height of the output frame use L1loss, and the attenuation factor is 0.1. the target frame bias uses L1loss, no attenuation. The total loss is the sum of the three types, and the detection result is shown in fig. 9.

Example 3

For different interlayer images of the SPECT or PET whole body tomographic image, the cenernet network model in example 2 can also be applied for detection of a lesion region.

A medical image classification and lesion area positioning system comprises a medical imaging device 1 and an image analysis server 2, wherein the medical imaging device 1 is in communication connection with the image analysis server 2;

the medical imaging device 1 comprises a screenshot module 11 and a sending module 12, wherein the screenshot module 11 is used for intercepting a medical image of the medical imaging device 1 in real time, and the sending module 12 is used for sending the medical image to the image analysis server 2;

the image analysis server 2 comprises a receiving module 21, a detection and classification module 22, a calling module 23, a sketching and labeling module 24 and a feedback sending module 25;

the receiving module 21 is configured to receive a medical image captured by the medical imaging device 1 in real time;

the detection classification module 22 is used for detecting and classifying the medical images and outputting detection classification results;

the calling module 23 is configured to call a detection positioning algorithm of a corresponding type of image when the detection classification result includes that the image scanning completion degree is 100%, to detect a potential organ of interest and a focus region, and to obtain detection information;

the delineation and annotation module 24 is used for delineation and annotation on the medical image according to the detection information to obtain an annotated auxiliary image;

the feedback sending module 25 is configured to send an auxiliary image to the medical imaging apparatus 1 in a feedback manner.

The system adopts a distributed architecture and consists of a medical image device 1 (client) and an image analysis server 2 (server), as shown in fig. 2. The functions and complexity of client software running on an operating computer of the medical imaging equipment 1 are simplified: the medical image of the medical imaging equipment 1 is intercepted in real time and uploaded to a service software end of the image analysis server 2, after the detection classification step and the detection positioning step of the image analysis server 2, client software of an operation computer of the medical imaging equipment 1 can obtain an auxiliary image with a hook and a label, which is fed back and sent by the service software end of the image analysis server 2, and the auxiliary image is clicked and displayed by an operator, so that influence and interference on the operation of the medical imaging equipment 1 on the operation computer are avoided.

In a further description, the medical imaging apparatus 1 further includes a timing module 13, and the timing module 13 is configured to periodically drive the screenshot module 11 to capture the medical image being scanned or being scanned in real time.

In the system of the invention, the medical image of any medical imaging device is captured in real time at regular time and is sent to the image analysis server 2 to form a training image, the image type and the position of the training image are artificially marked in the image analysis server 2, the training image is artificially shielded and the completion degree is marked for classification, and then the training image is input to the detection classification module 22 for detection and classification, and whether the detection and classification result is correct or not is checked. Therefore, the timing module 13 continuously learns and trains the training images of the detection and classification module 22, so that the detection and classification performance of the detection and classification module 22 is enhanced and a more efficient and accurate detection and classification effect is achieved.

To be further described, the medical image includes a scout image, or a whole body image;

the detection classification is to detect whether there is a medical image being scanned, and meanwhile, the completion degree of the medical image is evaluated according to a pre-designed completion degree classification standard, or the completion condition of the medical image is evaluated according to the analysis of the change contrast of the medical image obtained from the same medical imaging device 1 twice before and after.

To illustrate further, the detection classification module 22 is deployed with the algorithm of YOLO v 5.

The technical principle of the present invention is described above in connection with specific embodiments. The description is made for the purpose of illustrating the principles of the invention and should not be taken in any way as limiting the scope of the invention. Other embodiments of the invention will occur to those skilled in the art without the exercise of inventive faculty based on the explanations herein, and such equivalent modifications or substitutions are intended to be included within the scope of the present invention as defined in the appended claims.

Claims

1. A medical image classification and lesion area positioning method is characterized by comprising the following steps:

and 2, step: detecting and classifying the medical images;

2. The medical image classification and lesion region localization method according to claim 1, wherein the medical image comprises a scout image, a scout image or a whole body image;

3. The medical image classification and lesion region localization method of claim 1, wherein: the step 2 adopts the algorithm of YOLO v 5.

4. A medical image classification and lesion region localization method according to any one of claims 1-3, characterized in that: the detection positioning algorithm comprises a RetinaNet network module and a CenterNet network module;

5. A medical image classification and lesion region localization system, characterized by: the medical imaging equipment is in communication connection with the image analysis server;

the calling module is used for calling a detection positioning algorithm of the corresponding type of image when the detection classification result comprises that the image scanning completion degree is 100%, and detecting the potential interested organ and the focus area to obtain detection information;

6. A medical image classification and lesion region localization system according to claim 5, wherein: the medical image equipment further comprises a timing module, wherein the timing module is used for driving the screenshot module in a timing mode so as to capture the medical image which is being scanned or is scanned in real time.

7. A medical image classification and lesion region localization system according to claim 5, wherein: the medical image comprises a scout image, a scout image or a whole body image;

8. A medical image classification and lesion region localization system according to claim 5, wherein: the detection classification module 22 is deployed with the algorithm of YOLO v 5.

9. A medical image classification and lesion region localization system according to any one of claims 5-8, wherein: the detection positioning algorithm comprises a RetinaNet network module and a CenterNet network module;

10. A storage medium, characterized by: the storage medium is a computer readable storage medium having stored thereon a medical image classification and lesion region localization program, which when executed by a processor implements the steps of a medical image classification and lesion region localization method according to any one of claims 1 to 4.