WO2022247007A1

WO2022247007A1 - Medical image grading method and apparatus, electronic device, and readable storage medium

Info

Publication number: WO2022247007A1
Application number: PCT/CN2021/109482
Authority: WO
Inventors: 郭振; 柳杨; 李君�; 吕彬; 高艳
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-05-25
Filing date: 2021-07-30
Publication date: 2022-12-01
Also published as: CN113487621A

Abstract

A medical image grading method and apparatus, an electronic device, and a storage medium, relating to the technical field of artificial intelligence and digital medicine. The method comprises: acquiring a medical image to be graded, and performing feature extraction on said medical image using a feature extraction network in a pre-built lesion detection model to obtain a feature map (S1), wherein said medical image can be stored in a blockchain node; performing classification, recognition, and result statistics on the feature map to obtain a classification result (S2); performing region segmentation and area calculation on the feature map to obtain a segmentation result (S3); performing feature matching on the classification result and the segmentation result to obtain feature information (S4); grading said medical image using a pre-built first grading model to obtain a first grading result (S5); and performing grading correction on the feature information and the first grading result using a pre-built second grading model to obtain a target grading result (S6). The method can improve the accuracy of medical image grading.

Description

Medical image classification method, device, electronic equipment and readable storage medium

This application claims the priority of the Chinese patent application submitted to the China Patent Office on May 25, 2021, with the application number CN202110570809.8, and the title "Medical Image Extension Method, Device, Electronic Equipment, and Readable Storage Medium", all of which The contents are incorporated by reference in this application.

technical field

The present application relates to the fields of artificial intelligence and digital medical technology, and in particular to a medical image classification method, device, electronic equipment and readable storage medium.

Background technique

With the development of artificial intelligence, image recognition, as an important part of artificial intelligence, has been applied to various fields, such as applying image recognition in the medical field to identify medical images to judge the severity of diseases, such as grading fundus color ultrasound images, To determine the degree of diabetic retinopathy.

technical problem

However, the inventor realized that the current image classification method can only rely on a single image recognition model to classify medical images, and the feature dimensions are small, resulting in poor image classification accuracy.

technical solution

A medical image grading method provided by this application includes:

Acquiring the medical image to be classified, using the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map;

Carrying out classification recognition and result statistics on the feature map to obtain classification results;

Using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result;

performing feature matching on the classification result and the segmentation result to obtain feature information;

Classifying the medical image to be classified by using a pre-built first classification model to obtain a first classification result;

The feature information and the first grading result are graded and corrected by using the pre-built second grading model to obtain a target grading result.

The present application also provides a medical image grading device, the device comprising:

The feature matching module is used to obtain the medical image to be classified, and use the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map; perform classification recognition and result statistics on the feature map , obtaining a classification result; using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result; performing feature matching on the classification result and the segmentation result to obtain feature information;

An image classification module, configured to use a pre-built first classification model to classify the medical image to be classified to obtain a first classification result;

The grading correction module is configured to use the pre-built second grading model to perform grading correction on the feature information and the first grading result to obtain a target grading result.

The application also provides an electronic device, which includes:

a memory storing at least one computer program; and

A processor, executing the computer program stored in the memory to achieve the following steps:

The present application also provides a computer-readable storage medium, at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is executed by a processor in an electronic device to implement the following steps:

Description of drawings

FIG. 1 is a schematic flowchart of a medical image classification method provided by an embodiment of the present application;

Fig. 2 is a schematic module diagram of a medical image grading device provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of the internal structure of an electronic device implementing a method for grading medical images provided by an embodiment of the present application;

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The embodiments of the application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .

Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

In a medical application scenario, the sample image is a medical image, and the type of the object contained in the sample image is a lesion, that is, a part of the body where a lesion occurs. Medical imaging refers to internal tissues obtained in a non-invasive way for medical treatment or medical research, such as images of the stomach, abdomen, heart, knees, and brain, such as CT (Computed Tomography, computerized tomography), MRI ( Magnetic Resonance Imaging, Magnetic Resonance Imaging), US (ultrasonic, Ultrasound), X-ray images, EEG, and images generated by medical instruments with optical photography lights.

An embodiment of the present application provides a medical image classification method. The executor of the medical image grading method includes but is not limited to at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the medical image grading method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server can be an independent server, or it can provide cloud services, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network (ContentDelivery Network, CDN ), and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.

Referring to the schematic flowchart of the medical image classification method provided by an embodiment of the present application shown in FIG. 1, in the embodiment of the present application, the medical image classification method includes:

S1. Acquire the medical image to be classified, and use the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map;

Optionally, the medical image to be classified in the embodiment of the present application is a fundus color ultrasound image, and the lesion detection model includes: a feature extraction network, a lesion classification network, and a lesion segmentation network. Wherein, the feature extraction network is used for feature extraction, the lesion classification network is used for lesion classification, and the lesion segmentation network is used for lesion region segmentation.

In detail, in the embodiment of the present application, the initial feature extraction network in the feature extraction network is used to perform a convolution pooling operation on the medical image to be classified to obtain an initial feature map; the region extraction network in the feature extraction network is used to mark the The region of interest in the initial feature map is described to obtain the feature map.

Optionally, in the embodiment of the present application, the initial feature extraction network is a convolutional neural network, and the region extraction network is an RPN (Region Proposal Network, region proposal network).

Further, in the embodiment of the present application, before using the feature extraction network in the lesion detection model to perform feature extraction on the medical image to be classified, it also includes: acquiring a set of historical medical images, and performing preset labels on the set of historical medical images, Obtaining a first training image set; using the first training image set to iteratively train a pre-built first deep learning network model to obtain the lesion detection model. Wherein, the historical medical image set includes a plurality of historical medical images, and the historical medical images are medical images of the same type as the image to be classified but with different contents.

In detail, in this embodiment of the present application, preset labels are performed on the historical medical image set to obtain a first training image set, which includes: performing lesion area labeling on each historical medical image in the historical medical image set to obtain target area, performing lesion category marking on each target area in each of the historical medical images to obtain a first training image set; optionally, the preset lesion area includes a microvascular tumor area, a hemorrhage area, a hard Infiltration area, cotton wool spot area, laser spot area, neovascularization area, vitreous hemorrhage area, preretinal hemorrhage area, fibrous membrane area; the preset lesion category is in one-to-one correspondence with the preset lesion area, including: microvascular Tumor lesions, hemorrhage lesions, sclerotic lesions, cotton wool spot lesions, laser spot lesions, neovascular lesions, vitreous hemorrhage lesions, preretinal hemorrhage lesions, fibrous membrane lesions, if the target area is a laser spot area, then mark the area as a laser spot lesion.

Further, the lesion detection model is obtained by training the first deep model, therefore, the first deep learning model has the same network structure as the lesion detection model, therefore, the first deep learning model also includes : Feature extraction network, lesion classification network, lesion segmentation network.

In detail, in the embodiment of the present application, the first training image set is used to iteratively train the pre-built first deep learning network model to obtain the lesion detection model, wherein the first deep learning network model is Mask- RCNN model, including:

Step A: Use the feature extraction network in the first deep learning network model to perform convolution pooling on each image in the first training image set, and mark the region of interest on the images after convolution pooling , get the historical feature map;

Optionally, the feature extraction network in the embodiment of the present application includes an initial feature extraction network and a region extraction network; wherein, the initial feature extraction network is a convolutional neural network, and the region extraction network is an RPN (Region Proposal Network, Regional Advice Network).

In detail, in the embodiment of the present application, the initial feature extraction network is used to perform convolution pooling, and the region extraction network is used to mark the region of interest.

Step B: using the lesion classification network in the first deep learning network model to perform bounding box label prediction and classification prediction on the region of interest in the historical feature map, and obtain bounding box prediction coordinates and classification prediction values;

Step C: According to the lesion area marked by the historical feature image corresponding to the historical feature map, the real coordinates of the bounding box are obtained; according to the lesion category marked by the historical feature image corresponding to the historical feature map, the classification true value is obtained;

For example, if the marked lesion category is a laser spot lesion, then the true value of the classification of the corresponding laser spot lesion is 1.

Step D: Calculate according to the classification prediction value and the classification real value, using a preset first loss function to obtain a first loss value; according to the real coordinates of the bounding box and the predicted coordinates of the bounding box, use the preset Calculate the second loss function set to obtain the second loss function.

Optionally, the first loss function or the second loss function in this embodiment of the present application may be a cross-entropy loss function.

Optionally, the lesion segmentation network described in the embodiment of the present application includes a fully connected layer and a softmax network.

Step E: using the lesion segmentation network in the first deep learning network model to perform region segmentation prediction on the historical feature map, and obtain the predicted value of the total number of pixels corresponding to each region and the predicted value of the number of pixels at the edge of the region;

Optionally, the lesion segmentation network described in the embodiment of the present application is a fully convolutional network.

Step F: According to the lesion area marked by the historical feature image corresponding to the historical feature map, the real value of the total number of pixels in the corresponding area and the real value of the number of pixels at the edge of the area are obtained;

Step G: According to the predicted value of the total number of pixels in each region and the predicted value of the number of pixels at the edge of the region and the real value of the total number of pixels in the corresponding region and the real value of the number of pixels at the edge of the region, use the preset third loss function to calculate, obtaining a third loss value; summing the first loss value, the second loss value, and the third loss value to obtain a target loss value;

Optionally, in this embodiment of the present application, the third loss function is a cross-entropy loss function.

Step H: When the target loss value is greater than or equal to the preset loss threshold, update the first deep learning network model parameters, and return to step A above, until the target loss value is less than the preset loss threshold, stop training to obtain the lesion detection model.

In another embodiment of the present application, the high-throughput feature of the blockchain is used to store the medical images to be graded in the blockchain nodes to improve data access efficiency.

S2. Perform classification recognition and result statistics on the feature map to obtain classification results;

In detail, in the embodiment of the present application, the lesion segmentation network in the lesion detection model is used to mark and classify the bounding boxes of the feature map, and the number of bounding boxes of the same category is summed up to obtain the classification result. For example: there are 4 bounding boxes A, B, C, and D in the feature map, the A bounding box is classified as a hemorrhage lesion, the B bounding box is a laser spot lesion, the C bounding box is a preretinal hemorrhage lesion, and the D bounding box is For bleeding lesions, then the number of bounding boxes of the same category is collected to obtain the classification result. There are two bounding boxes A and D bounding boxes for bleeding lesions, one bounding box B for laser spot lesions, and one preretinal hemorrhage lesion is the bounding box of C.

S3. Perform region segmentation and area calculation on the feature map to obtain a segmentation result;

In detail, in the embodiment of the present application, the lesion segmentation network in the lesion detection model is used to segment the feature map to obtain multiple segmentation regions. Optionally, the lesion segmentation network in the embodiment of the application is a full Convolutional network, further, since the size of the segmented regions corresponding to images of different sizes to be classified is quite different, in order to facilitate comparison, it is necessary to unify the comparison standard, calculate the area ratio of each segmented region to the medical image to be classified, and obtain the corresponding The relative area is not affected by the area change of the medical image to be classified; all the segmented regions and the relative area corresponding to each segmented region are summed up to obtain the segmented result. For example: the segmentation result is that there are 4 segmentation regions A, B, C, and D in the feature map, the segmentation region A consists of 10 pixels, and the medical image to be classified consists of 100 pixels, then the corresponding relative The area is 10%.

S5. Perform feature matching on the classification result and the segmentation result to obtain feature information;

In detail, the embodiment of the present application matches and associates the classification result with the segmentation result to obtain the lesion category corresponding to each relative area in the segmentation result.

Specifically, the classification result and the segmentation result in the embodiment of the present application are obtained from different branches in the same large model, and the positions of each bounding box and segmentation region in the classification result are the same, for example, the classification result There is a bounding box A for the bleeding lesions, and the segmented area a corresponds to the bounding box A. Therefore, the corresponding lesion category of the segmented area a obtained by matching is the bleeding lesion.

Further, in the embodiment of the present application, all the relative areas corresponding to the same lesion category in the segmentation results are summed to obtain the corresponding total area of the segmented region; the total area of the segmented region is combined with the corresponding lesion category, Obtain a matching array, for example: in the segmentation result, the segmented areas corresponding to the preretinal hemorrhage lesion category are A and B, the relative area corresponding to the A segmented area is 10%, and the relative area corresponding to the B segmented area is 20%, then the preretinal The total area of each area corresponding to the category of hemorrhage lesions is (10%+20%)=30%, and the corresponding matching array is [preretinal hemorrhage lesions, 30%]; further, in the embodiment of the present application, all matching arrays are randomly combination to obtain the feature information.

S4. Using a grading model to classify the medical image to be classified to obtain a first grading result;

In detail, in the embodiment of the present application, the classification model is used to classify the medical images to be classified, and before obtaining the first classification result, it also includes: performing preset classification labels on the historical medical image set to obtain the second training image set; using the second training image set to iteratively train the pre-built second deep learning network model to obtain the first hierarchical model. Optionally, the grading labels include: mild non-proliferative retinopathy, moderate non-proliferative retinopathy, severe non-proliferative retinopathy, proliferative retinopathy and normal fundus.

Optionally, the second deep learning network model in the embodiment of the present application is a convolutional neural network model including a dense attention mechanism.

S6. Classify the feature information by using the second classification model to obtain a target classification result.

Optionally, in this embodiment of the present application, the second hierarchical model is a random forest network model.

Further, in the embodiment of the present application, in order to make the classification result more accurate, the first classification result needs to be corrected. Therefore, the embodiment of the present application uses the target classification network model to classify the feature information to obtain the target classification result .

In detail, before using the target classification network to classify the feature information in the embodiment of the present application, it also includes: using the preset lesion category label as the root node, using the pre-built relative area classification interval and the preset classification Labels are used as classification conditions, and a random forest model is constructed to obtain the second grading model, wherein the grading labels share mild non-proliferative retinopathy, moderate non-proliferative retinopathy, severe non-proliferative retinopathy, proliferative retinopathy There are five kinds of lesions and normal fundus, and the classification interval of the lesion area can be set according to the actual diagnosis experience, such as [0, 20%), [20%, 40%), [40%, 60%), [60% %, 80%), [80%, 100%].

Further, in the embodiment of the present application, the feature information and the first grading result are input into the second grading model to obtain the target grading result, for example: the first grading result is moderate non-proliferative retina The feature information of the lesion is [preretinal hemorrhage lesion, 10%], then the first grading result and the feature information are input into the second grading model, and the target grading result is mild non-proliferative retinopathy.

As shown in FIG. 3 , it is a functional block diagram of the medical image grading device of the present application.

The medical image grading apparatus 100 described in this application can be installed in an electronic device. According to the realized functions, the medical image grading device may include a feature matching module 101, an image grading module 102, and a grading correction module 103. The module described in the present invention may also be referred to as a unit, which refers to a device that can be processed by an electronic device processor. A series of computer program segments executed and capable of performing a fixed function, stored in the memory of an electronic device.

In this embodiment, the functions of each module/unit are as follows:

The feature matching module 101 is used to obtain the medical image to be classified, and use the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map; classify and identify the feature map and The results are counted to obtain classification results; using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain segmentation results; performing feature matching between the classification results and the segmentation results to obtain features information;

In detail, the feature matching module 101 in the embodiment of the present application uses the initial feature extraction network in the feature extraction network to perform a convolution pooling operation on the medical image to be classified to obtain an initial feature map; The region extraction network marks the region of interest in the initial feature map to obtain a feature map.

Further, before the feature matching module 101 in the embodiment of the present application utilizes the feature extraction network in the lesion detection model to perform feature extraction on the medical image to be classified, it also includes: acquiring a historical medical image set, and analyzing the historical medical image set Preset labeling is performed to obtain a first training image set; and the pre-built first deep learning network model is iteratively trained by using the first training image set to obtain the lesion detection model. Wherein, the historical medical image set includes a plurality of historical medical images, and the historical medical images are medical images of the same type as the image to be classified but with different contents.

In detail, the feature matching module 101 in the embodiment of the present application performs preset labeling on the historical medical image set to obtain the first training image set, including: for each lesion in the historical medical image set Marking the lesion area to obtain a target area, and performing lesion category marking on each of the target areas in each of the historical medical images to obtain a first training image set; optionally, the preset lesion area includes a microvascular tumor area, hemorrhage area, sclerotic area, cotton wool spot area, laser spot area, neovascular area, vitreous hemorrhage area, preretinal hemorrhage area, fibrous membrane area; the preset lesion category and the preset lesion area are one by one Correspondingly, including: microvascular tumor lesions, bleeding lesions, sclerotic lesions, cotton wool spot lesions, laser spot lesions, neovascular lesions, vitreous hemorrhage lesions, preretinal hemorrhage lesions, and fibrous membrane lesions. If the target area is a laser spot area, then the This area is marked as a laser spot lesion.

In detail, in the embodiment of the present application, the feature matching module 101 uses the first training image set to iteratively train the pre-built first deep learning network model to obtain the lesion detection model, wherein the first depth The learning network model is the Mask-RCNN model, including:

In detail, the feature matching module 101 in the embodiment of the present application uses the lesion segmentation network in the lesion detection model to mark and classify the bounding boxes of the feature maps, and summarize the number of bounding boxes of the same category, Get classification results. For example: there are 4 bounding boxes A, B, C, and D in the feature map, the A bounding box is classified as a hemorrhage lesion, the B bounding box is a laser spot lesion, the C bounding box is a preretinal hemorrhage lesion, and the D bounding box is For bleeding lesions, then the number of bounding boxes of the same category is collected to obtain the classification result. There are two bounding boxes A and D bounding boxes for bleeding lesions, one bounding box B for laser spot lesions, and one preretinal hemorrhage lesion is the bounding box of C.

In detail, the feature matching module 101 in the embodiment of the present application uses the lesion segmentation network in the lesion detection model to perform region segmentation on the feature map to obtain multiple segmented regions. Optionally, the The lesion segmentation network described above is a fully convolutional network. Further, since the size of the segmented regions corresponding to images of different sizes to be classified is quite different, in order to facilitate comparison, a unified comparison standard is required to calculate the difference between each segmented region and the medical image to be classified. The corresponding relative area is obtained, and the relative area is not affected by the area change of the medical image to be classified; all the segmented regions and the relative area corresponding to each segmented region are summarized to obtain the segmented result. For example: the result of the segmentation is that there are 4 segmentation regions A, B, C, and D in the feature map, the segmentation region A consists of 10 pixels, and the medical image to be classified consists of 100 pixels, then the corresponding relative The area is 10%.

In detail, the feature matching module 101 in the embodiment of the present application matches and associates the classification result with the segmentation result to obtain the lesion category corresponding to each relative area in the segmentation result.

Further, the feature matching module 101 in the embodiment of the present application sums all the relative areas corresponding to the same lesion category in the segmentation results to obtain the corresponding total area of the segmented area; compares the total area of the segmented area with the corresponding The lesion categories are combined to obtain a matching array, for example: in the segmentation result, the segmentation areas corresponding to the preretinal hemorrhage lesion category are A and B, the relative area corresponding to the A segmentation area is 10%, and the relative area corresponding to the B segmentation area is 20%, then the total area of each region corresponding to the preretinal hemorrhage lesion category is (10%+20%)=30%, and the corresponding matching array is [preretinal hemorrhage lesion, 30%]; further, the embodiment of the present application Randomly combine all matching arrays to obtain the feature information.

The image classification module 102 is configured to use a pre-built first classification model to classify the medical image to be classified to obtain a first classification result;

In detail, in the embodiment of the present application, the image classification module 102 uses a classification model to classify the medical images to be classified, and before obtaining the first classification result, it also includes: performing preset classification labels on the historical medical image set , to obtain a second training image set; using the second training image set to iteratively train the pre-built second deep learning network model to obtain the first hierarchical model. Optionally, the grading labels include: mild non-proliferative retinopathy, moderate non-proliferative retinopathy, severe non-proliferative retinopathy, proliferative retinopathy and normal fundus.

The grading correction module 103 is configured to use the pre-built second grading model to perform grading correction on the feature information and the first grading result to obtain a target grading result.

Furthermore, in this embodiment of the present application, in order to make the classification result more accurate, the first classification result needs to be corrected. Therefore, the classification correction module 103 in the embodiment of the present application uses the target classification network model to classify the feature information, Obtain the target classification result.

In detail, before the grading correction module 103 in the embodiment of the present application uses the target grading network to classify the feature information, it also includes: using the preset lesion category label as the root node, using the pre-built relative area classification Intervals and preset grading labels are used as classification conditions to construct a random forest model to obtain the second grading model, wherein the grading labels share mild non-proliferative retinopathy, moderate non-proliferative retinopathy, severe non-proliferative retinopathy There are five types of retinopathy, proliferative retinopathy, and normal fundus. The lesion area classification interval can be set according to actual diagnosis experience, such as [0, 20%), [20%, 40%), [40%, 60%), [60%, 80%), [80%, 100%].

Further, in the embodiment of the present application, the grading correction module 103 inputs the feature information and the first grading result into the second grading model to obtain the target grading result, for example: the first grading result is Moderate non-proliferative retinopathy, the feature information is [preretinal hemorrhage lesion, 10%], then input the first grading result and the feature information into the second grading model, and the target grading result is mild non-proliferative Retinopathy.

As shown in FIG. 3 , it is a schematic structural diagram of an electronic device implementing the medical image classification method of the present application.

The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and operable on the processor 10, such as a medical image grading program .

Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, CD etc. The storage 11 may be an internal storage unit of the electronic device in some embodiments, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD ) card, flash memory card (Flash Card), etc. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various data installed in the electronic device, such as codes of medical image grading programs, but also can be used to temporarily store outputted or to-be-outputted data.

In some embodiments, the processor 10 may be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combination of central processing unit (Central Processing unit, CPU), microprocessor, digital processing chip, graphics processor and various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and runs or executes programs or modules stored in the memory 11 (such as medical image grading program, etc.), and call the data stored in the memory 11 to execute various functions of the electronic device and process data.

The communication bus 12 can be a peripheral component interconnection standard (perIPheral component interconnect (PCI for short) bus or extended industry standard structure (extended industry standard architecture, referred to as EISA) bus and so on. The bus can be divided into address bus, data bus, control bus and so on. The communication bus 12 is configured to realize connection and communication between the memory 11 and at least one processor 10 and the like. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

Figure 3 only shows an electronic device with components, and those skilled in the art can understand that the structure shown in Figure 3 does not constitute a limitation to the electronic device, and may include fewer or more components than shown in the figure , or combinations of certain components, or different arrangements of components.

For example, although not shown, the electronic device may also include a power supply (such as a battery) for supplying power to various components. Preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that Realize functions such as charge management, discharge management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components. The electronic device may also include various sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be repeated here.

Optionally, the communication interface 13 may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which are generally used to establish a communication connection between the electronic device and other electronic devices.

Optionally, the communication interface 13 may also include a user interface, the user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface . Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like. Wherein, the display may also be properly referred to as a display screen or a display unit, and is used for displaying information processed in the electronic device and for displaying a visualized user interface.

It should be understood that the embodiments are only for illustration, and are not limited by the structure in terms of the scope of the patent application.

The medical image grading program stored in the memory 11 in the electronic device is a combination of multiple computer programs. When running in the processor 10, it can realize:

Specifically, for a specific implementation method of the above computer program by the processor 10, reference may be made to the description of relevant steps in the embodiment corresponding to FIG. 1 , and details are not repeated here.

Furthermore, if the integrated modules/units of the electronic equipment are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer readable medium may be non-volatile or volatile. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) .

The embodiment of the present application can also provide a computer-readable storage medium, the readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, it can realize:

Further, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function, etc.; use of the created data, etc.

In the several embodiments provided in this application, it should be understood that the disclosed devices, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

Therefore, the embodiments should be regarded as exemplary and not restrictive in all points of view, and the scope of the application is defined by the appended claims rather than the foregoing description, and it is intended that the scope of the present application be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in this application. Any reference sign in a claim should not be construed as limiting the claim concerned.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain (Blockchain), essentially a decentralized database, is a series of data blocks associated with each other using cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or devices stated in the system claims may also be realized by one unit or device through software or hardware. Secondary terms are used to denote names without implying any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application without limitation. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be Make modifications or equivalent replacements without departing from the spirit and scope of the technical solutions of the present application.

Claims

A method for grading medical images, wherein the method includes:

Acquiring the medical image to be classified, using the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map;

Carrying out classification recognition and result statistics on the feature map to obtain classification results;

Using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result;

performing feature matching on the classification result and the segmentation result to obtain feature information;

Classifying the medical image to be classified by using a pre-built first classification model to obtain a first classification result;

The feature information and the first grading result are graded and corrected by using the pre-built second grading model to obtain a target grading result.
The method for grading medical images according to claim 1, wherein said using the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map, comprising:

performing a convolution pooling operation on the medical image to be classified to obtain an initial feature map;

A region of interest in the initial feature map is marked to obtain a feature map.
The medical image classification method according to claim 1, wherein, before performing feature extraction on the medical image to be classified by using the feature extraction network in the pre-built lesion detection model, the method further comprises:

Acquiring a set of historical medical images, and performing label marking on the set of historical medical images to obtain a first set of training images;

The first training image set is used to iteratively train the pre-built first deep learning network model to obtain the lesion detection model.
The medical image classification method according to claim 3, wherein said labeling said historical medical image set comprises:

performing lesion area division on the lesions in each historical medical image in the historical medical image set to obtain a target area;

Each of the target regions in each of the historical medical images is labeled with a lesion category by using the preset lesion category label.
The method for grading medical images according to claim 3 or 4, wherein said using the pre-built first grading model to classify the medical image to be graded, before obtaining the first grading result, said method further comprises:

performing preset classification labels on the historical medical image set to obtain a second training image set;

The second training image set is used to iteratively train the pre-built second deep learning network model to obtain the first hierarchical model.
The medical image grading method according to claim 1, wherein said using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result, comprising:

performing region segmentation on the feature map to obtain a plurality of segmented regions;

calculating the area ratio of each segmented region to the medical image to be classified to obtain the relative area corresponding to the segmented region;

Summarize all the segmented regions and the relative area corresponding to each segmented region to obtain the segmented result.
The medical image classification method according to claim 6, wherein said performing feature matching on said classification result and said segmentation result to obtain feature information comprises:

Matching and associating the classification result with the segmentation result to obtain a lesion category corresponding to each relative area in the segmentation result;

Summing all the relative areas corresponding to the same lesion category in the segmentation result to obtain the total area of the corresponding segmented region;

Combining the total area of the segmented regions with the corresponding lesion category to obtain a matching array;

Randomly combine all matching arrays to obtain the feature information.
A medical image grading device, including:

The feature matching module is used to obtain the medical image to be classified, and use the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map; perform classification recognition and result statistics on the feature map , obtaining a classification result; using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result; performing feature matching on the classification result and the segmentation result to obtain feature information;

An image classification module, configured to use a pre-built first classification model to classify the medical image to be classified to obtain a first classification result;

The grading correction module is configured to use the pre-built second grading model to perform grading correction on the feature information and the first grading result to obtain a target grading result.
An electronic device, wherein the electronic device includes:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor, the computer program is executed by the at least one processor, so that the at least one processor can perform the following steps:

Acquiring the medical image to be classified, using the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map;

Carrying out classification recognition and result statistics on the feature map to obtain classification results;

Using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result;

performing feature matching on the classification result and the segmentation result to obtain feature information;

Classifying the medical image to be classified by using a pre-built first classification model to obtain a first classification result;

The feature information and the first grading result are graded and corrected by using the pre-built second grading model to obtain a target grading result.
The electronic device according to claim 9, wherein the feature extraction network of the pre-built lesion detection model is used to perform feature extraction on the medical image to be classified to obtain a feature map, comprising:

performing a convolution pooling operation on the medical image to be classified to obtain an initial feature map;

A region of interest in the initial feature map is marked to obtain a feature map.
The electronic device according to claim 10, wherein, before performing feature extraction on the medical image to be classified using the feature extraction network in the pre-built lesion detection model, the method further comprises:

Acquiring a set of historical medical images, and performing label marking on the set of historical medical images to obtain a first set of training images;

The first training image set is used to iteratively train the pre-built first deep learning network model to obtain the lesion detection model.
The electronic device of claim 11, wherein said tagging said set of historical medical images comprises:

performing lesion area division on the lesions in each historical medical image in the historical medical image set to obtain a target area;

Each of the target regions in each of the historical medical images is labeled with a lesion category by using the preset lesion category label.
The electronic device according to claim 11 or 12, wherein, before the first classification model is used to classify the medical image to be classified, before obtaining the first classification result, the method further comprises:

performing preset classification labels on the historical medical image set to obtain a second training image set;

The second training image set is used to iteratively train the pre-built second deep learning network model to obtain the first hierarchical model.
The electronic device according to claim 9, wherein said using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result, comprising:

performing region segmentation on the feature map to obtain a plurality of segmented regions;

calculating the area ratio of each segmented region to the medical image to be classified to obtain the relative area corresponding to the segmented region;

Summarize all the segmented regions and the relative area corresponding to each segmented region to obtain the segmented result.
The electronic device according to claim 14, wherein said performing feature matching on said classification result and said segmentation result to obtain feature information comprises:

Matching and associating the classification result with the segmentation result to obtain a lesion category corresponding to each relative area in the segmentation result;

Summing all the relative areas corresponding to the same lesion category in the segmentation result to obtain the total area of the corresponding segmented region;

Combining the total area of the segmented regions with the corresponding lesion category to obtain a matching array;

Randomly combine all matching arrays to obtain the feature information.
A computer-readable storage medium storing a computer program, wherein the computer program implements the following steps when executed by a processor:

Acquiring the medical image to be classified, using the feature extraction network in the pre-built lesion detection model to perform feature extraction on the medical image to be classified to obtain a feature map;

Carrying out classification recognition and result statistics on the feature map to obtain classification results;

Using the lesion segmentation network in the lesion detection model to perform region segmentation and area calculation on the feature map to obtain a segmentation result;

performing feature matching on the classification result and the segmentation result to obtain feature information;

Classifying the medical image to be classified by using a pre-built first classification model to obtain a first classification result;

The feature information and the first grading result are graded and corrected by using the pre-built second grading model to obtain a target grading result.
The computer-readable storage medium according to claim 16, wherein the feature extraction network of the pre-built lesion detection model is used to perform feature extraction on the medical image to be classified to obtain a feature map, comprising:

performing a convolution pooling operation on the medical image to be classified to obtain an initial feature map;

A region of interest in the initial feature map is marked to obtain a feature map.
The computer-readable storage medium according to claim 16, wherein, before performing feature extraction on the medical image to be classified using the feature extraction network in the pre-built lesion detection model, the method further comprises:

Acquiring a set of historical medical images, and performing label marking on the set of historical medical images to obtain a first set of training images;

The first training image set is used to iteratively train the pre-built first deep learning network model to obtain the lesion detection model.
The computer-readable storage medium of claim 18, wherein said tagging said set of historical medical images comprises:

performing lesion area division on the lesions in each historical medical image in the historical medical image set to obtain a target area;

Each of the target regions in each of the historical medical images is labeled with a lesion category by using the preset lesion category label.
The computer-readable storage medium according to claim 18 or 19, wherein, before the said medical image to be graded is graded using the pre-built first grade model, and before the first grade result is obtained, the method further comprises:

performing preset classification labels on the historical medical image set to obtain a second training image set;

The second training image set is used to iteratively train the pre-built second deep learning network model to obtain the first hierarchical model.