WO2022179083A1

WO2022179083A1 - Image detection method and apparatus, and device, medium and program

Info

Publication number: WO2022179083A1
Application number: PCT/CN2021/117801
Authority: WO
Inventors: 孙辉; 韩泓泽; 刘星龙; 黄宁; 张少霆
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-02-25
Filing date: 2021-09-10
Publication date: 2022-09-01
Also published as: JP2023518160A; CN112949654A

Abstract

Disclosed in embodiments of the present application are an image detection method and apparatus, and a device, a computer-readable storage medium and a computer program, the image detection method comprising: acquiring a medical image to be subjected to detection; performing feature extraction on said medical image so as to obtain first feature maps of several dimensions; using a first feature map of a preset dimension as a reference feature map, and generating a lesion probability graph by using the reference feature map, wherein the lesion probability graph is used to represent the probabilities of different regions in said medical image having lesions; fusing the lesion probability graph with the first feature maps of the several dimensions so as to obtain a final fused feature map; and performing detection processing on the final fused feature map, so as to obtain a detection result of lesions in said medical image.

Description

Image detection method, apparatus, equipment, medium and program

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the Chinese patent application with the application number of 202110214861.X and the filing date of February 25, 2021, entitled "Image Detection Method and Related Apparatus and Equipment", and claims the priority of the Chinese patent application. The entire contents of the patent application are incorporated herein by reference.

technical field

The present application relates to the technical field of artificial intelligence, and relates to, but is not limited to, an image detection method, apparatus, device, computer-readable storage medium, and computer program.

Background technique

Medical images such as Computed Tomography (CT) are of great clinical significance. For example, doctors can find organ lesions such as pneumonia through medical images. And with the development of information technology, electronic devices with processing capabilities such as computers are gradually replacing manual tasks in all walks of life. In the field of clinical application, by using electronic equipment to detect medical images, the detection results of lesions in the medical images can be obtained to assist doctors in clinical practice.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image detection method, apparatus, device, computer-readable storage medium, and computer program.

An embodiment of the present application provides an image detection method, including: acquiring a medical image to be tested; performing feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; using the first feature map of a preset dimension as a reference feature Figure, using the reference feature map to generate a lesion probability map, where the lesion probability map is used to represent the probability that different areas in the medical image to be tested belong to the lesion; the lesion probability map is fused with the first feature maps of several dimensions to obtain the final Fusion feature map; detection processing is performed on the final fused feature map to obtain the detection result of the lesion in the medical image to be tested.

Therefore, by performing feature extraction on the obtained medical image to be tested, a first feature map of several dimensions is obtained, and the first feature map of a preset dimension is used as a reference feature map, so that the reference feature map is used to generate a lesion probability map, And the lesion probability map is used to represent the probability that different areas in the medical image to be tested belong to the lesion, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain the final fusion feature map, so the lesion probability map can be used as a global The features are fused with the first feature map, so that the final fusion feature map can enhance the specificity of the lesion, and then the detection result of the lesion in the medical image to be tested can be obtained by performing detection processing through the final fusion feature map, which can improve image detection. accuracy.

Wherein, before using the reference feature map to generate the lesion probability map, the method further includes: using the reference feature map to perform prediction processing to obtain a first probability value of the lesion contained in the medical image to be tested; and determining whether to perform the use of the reference feature based on the first probability value. Steps to generate a lesion probability map and subsequent steps.

Therefore, by using the reference feature map for prediction processing, the first probability value of the lesion contained in the medical image to be tested is obtained, and based on the first probability value, it is determined whether to execute the step of using the reference feature map to generate the lesion probability map and the subsequent steps, so as to It is beneficial to solve the problem of false positive detection results obtained when the medical image to be tested does not contain lesions, which can further improve the accuracy of image detection, and because the negative data can be pre-screened before detection, it can improve the image quality. detection efficiency.

Wherein, based on the first probability value, determining whether to perform the step of using the reference feature map to generate the lesion probability map and subsequent steps includes: if the first probability value satisfies the first preset condition, performing the step of using the reference feature map to generate the lesion probability map step and subsequent steps; or, when the medical image to be tested is a two-dimensional medical image included in a three-dimensional medical image; based on the first probability value, determine whether to perform the step of using the reference feature map to generate the lesion probability map and the subsequent steps, The method includes: sorting the first probability values corresponding to the two-dimensional medical images in descending order, and selecting a first preset number of first probability values; performing preset processing on the preset number of first probability values to obtain the second probability value; if the second probability value satisfies the second preset condition, the step of generating the lesion probability map by using the reference feature map and the subsequent steps are performed.

Therefore, when the first probability value satisfies the first preset condition, the step of generating the lesion probability map by using the reference feature map and the subsequent steps are performed, or, in the case where the medical image to be tested is a two-dimensional medical image included in a three-dimensional medical image Next, sort the first probability values of the two-dimensional medical images in descending order, select the first preset number of first probability values, and perform preset processing on the preset number of first probability values to obtain the second probability value, so that when the second probability value satisfies the second preset condition, the step of using the reference feature map to generate the probability map of the lesion and the subsequent steps are performed, which can help to pre-screen negative data before detection, thereby improving the image quality. Detection accuracy and efficiency.

Wherein, the first preset condition includes: the first probability value is greater than or equal to the first probability threshold; the second preset condition includes: the second probability value is greater than or equal to the second probability threshold; and the preset processing is an average operation.

Therefore, by setting the first preset condition such that the first probability value is greater than or equal to the first probability threshold, and by setting the second preset condition such that the second probability value is greater than or equal to the second probability threshold, the preset process is set to average Therefore, when the first probability value is greater than or equal to the first probability threshold, the use of The step of generating a case probability map with reference to the feature map and the subsequent steps, when the second probability value is greater than or equal to the second probability threshold, the step of generating a case probability map by using the reference feature map and the subsequent steps are performed, so it can be beneficial to pre-detection before detection. Screen out negative data to improve the accuracy and efficiency of image detection.

Wherein, if the first probability value does not meet the first preset condition or the second probability value does not meet the second preset condition, it is determined that the medical image to be tested does not contain a lesion.

Therefore, when the first probability value does not meet the first preset condition or the second probability value does not meet the second preset condition, it is determined that the medical image to be tested does not contain a lesion, so that the user can timely perceive the negative of the medical image to be tested. The detection result can be beneficial to improve the user experience.

Wherein, using the reference feature map to generate the lesion probability map includes: counting the gradient values of each pixel in the reference feature map with respect to the lesion, generating a class activation map, and using the class activation map as the lesion probability map.

Therefore, by counting the gradient values of each pixel in the reference feature map with respect to the lesion, a class activation map is generated to serve as the lesion probability map, which can improve the accuracy of the lesion probability map, thereby improving the accuracy of subsequent image detection.

Wherein, fusing the lesion probability map with the first feature maps of several dimensions to obtain a final fusion feature map includes: encoding the reference feature map by using the lesion probability map to obtain a second feature map; combining the second feature map with The first feature maps of several dimensions are fused to obtain the final fused feature map.

Therefore, by using the lesion probability map to encode the reference feature map, a second feature map is obtained, and the second feature map is fused with the first feature maps of several dimensions to obtain the final fusion feature map. The probability map is used as a global feature to participate in the feature map fusion, so that the final fusion feature map can enhance the specificity of the lesion, which can help to improve the accuracy of subsequent image detection.

Wherein, using the lesion probability map to encode the reference feature map to obtain a second feature map, comprising: comparing the pixel value of the first pixel point in the lesion probability map with the second pixel point corresponding to the first pixel point in the reference feature map Multiply the pixel values of , to obtain the pixel values of the corresponding pixel points of the second feature map.

Therefore, by multiplying the pixel value of the first pixel in the lesion probability map with the pixel value of the second pixel corresponding to the first pixel in the reference feature map, the pixel value of the corresponding pixel in the second feature map is obtained, In this way, the coding processing of the reference feature map by the lesion probability map is realized, which can help to reduce the amount of calculation.

Among them, the second feature map is fused with the first feature maps of several dimensions to obtain the final fusion feature map, which includes: according to the order of dimensions from high to bottom, the second feature map and the first feature map of each dimension sorted in order A feature map is fused to obtain the final fused feature map.

Therefore, by merging the second feature map with the first feature map of each dimension in the order of dimensions from high to low to obtain the final fused feature map, it is beneficial to perform feature map fusion dimension by dimension, which can be beneficial to Fully fuse context information to improve the accuracy and feature richness of the final fused feature map, which in turn can help improve the accuracy of subsequent image detection.

Among them, the reference feature map is the first feature map with the highest dimension; according to the order of dimensions from high to bottom, the second feature map is fused with the first feature map of each dimension sorted in order to obtain the final fusion feature map, including : fuse the reference feature map with the first low-dimensional feature map to obtain a first fusion feature map with the same dimension as the first low-dimensional feature map, wherein the first low-dimensional feature map is one dimension lower than the reference feature map The first feature map; the second feature map is fused with the first fusion feature map to obtain a second fusion feature map with the same dimension as the first fusion feature map; the second fusion feature map and the second low-dimensional feature are repeatedly performed. The graphs are fused to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused; wherein, the second low-dimensional feature map is smaller than the current second fused feature map. The first feature map with one dimension lower than the feature map; the second fused feature map obtained by the final fusion is used as the final fused feature map.

Therefore, by fusing the reference feature map with the first low-dimensional feature map, a first fused feature map with the same dimension as the first low-dimensional feature map is obtained, and the first low-dimensional feature map is one dimension lower than the reference feature map , and fuse the second feature map with the first fusion feature map to obtain a second fusion feature map with the same dimension as the first fusion feature map. The low-dimensional feature maps are fused to obtain a new second fusion feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, and the second low-dimensional feature map is smaller than the current first feature map. The second fusion feature map is one dimension lower than the first feature map, and the second fusion feature map obtained by final fusion is used as the final fusion feature map, and then the lesion probability map can be used as a global feature to couple with the decoding process of image detection, so that the final The fusion feature map can enhance the specificity of the lesion, and can fully fuse the context information of the feature map to improve the accuracy and feature richness of the final fused feature map, which in turn can help improve the accuracy of subsequent image detection.

The detection result includes the detection area of the lesion in the medical image to be tested; the method further includes: performing organ detection on the medical image to be tested to obtain the organ area in the medical image to be tested; obtaining the lesion occupied by the detection area of the lesion in the organ area Proportion.

Therefore, by performing organ detection on the medical image to be tested, the organ region in the medical image to be tested can be obtained, and the proportion of the lesion occupied by the detection region of the lesion in the organ region can be obtained. reference information, so as to improve user experience.

Wherein, before the feature extraction is performed on the medical image to be tested and the first feature maps of several dimensions are obtained, the method further includes: preprocessing the medical image to be tested, wherein the operation of preprocessing at least includes: using a preset window value to The pixel values of the medical image are normalized to a predetermined range.

Therefore, before the feature extraction is performed on the medical image to be tested, the medical image to be tested is preprocessed, and the preprocessing operation at least includes: using a preset window value to normalize the pixel value of the medical image to be tested to a preset range, The contrast of the medical image to be tested can be enhanced, and the accuracy of the subsequently extracted first feature map can be improved.

Among them, performing feature extraction on the medical image to be tested to obtain first feature maps of several dimensions, including: using the feature extraction sub-network of the image detection model to perform feature extraction on the medical image to be tested to obtain first feature maps of several dimensions; The lesion probability map is fused with the first feature maps of several dimensions to obtain a final fusion feature map, including: using the fusion processing sub-network of the image detection model to fuse the lesion probability map with the first feature maps of several dimensions to obtain the final Fusion feature map; performing detection processing on the final fused feature map to obtain detection results about lesions in the medical image to be tested, including: using the fusion processing sub-network of the image detection model to detect and process the final fused feature map to obtain the medical image to be tested The detection results of the lesions in .

Therefore, by using the feature extraction sub-network of the image detection model to extract the features of the medical image to be tested, the first feature maps of several dimensions are obtained, and the fusion processing sub-network of the image detection model is used to combine the lesion probability map with the first feature maps of several dimensions. The feature maps are fused to obtain the final fused feature map, and the fusion processing sub-network of the image detection model is used to detect and process the final fused feature map to obtain the detection results of the lesions in the medical image to be tested, so as to perform feature extraction through the image detection model. , fusion processing, and image detection tasks, which can help improve the efficiency of image detection.

Wherein, before using the feature extraction sub-network of the image detection model to perform feature extraction on the medical image to be tested, and before obtaining the first feature maps of several dimensions, the method further includes: acquiring a sample medical image, wherein the sample medical image includes the actual area of the lesion ; Use the feature extraction sub-network to perform feature extraction on the sample medical image, and obtain the first sample feature map of several dimensions; take the first sample feature map of the preset dimension as the reference sample feature map, and use the reference sample feature map to generate lesions The sample probability map, in which the lesion sample probability map is used to represent the probability that different regions of the sample medical image belong to the lesion; the fusion processing sub-network is used to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain the final Fuse the sample feature maps; use the fusion processing sub-network to detect and process the final fused sample feature maps to obtain the detection area of the lesion in the sample medical image; use the difference between the actual area and the detection area to adjust the network parameters of the image detection model.

Therefore, by acquiring a sample medical image, and the sample medical image contains the actual area of the lesion, and using the feature extraction sub-network to perform feature extraction on the sample medical image, a first sample feature map of several dimensions is obtained, so that the preset dimension The first sample feature map is used as the reference sample feature map, and the reference sample feature map is used to generate the lesion sample probability map, and the lesion sample probability map is used to represent the probability that different areas of the sample medical image belong to the lesion, and then the lesion probability map is combined with the lesion probability map. The first sample feature maps of several dimensions are fused to obtain the final fused sample feature map, so as to perform detection processing on the final fused sample feature map to obtain the detection area of the lesion in the sample medical image, and use the actual area and the detection area. Therefore, in the training process of the image detection model, the probability map of the lesion sample can be used as a global feature to couple with the decoding process of image detection, so that the final fusion sample feature map can strengthen the detection of lesions. Therefore, the sensitivity of the image detection model to the lesions can be enhanced, and the training speed of the model can be improved.

Among them, using the difference between the actual area and the detection area to adjust the network parameters of the feature extraction sub-network and the fusion processing sub-network includes: using the set similarity loss function to process the actual area and the detection area, and determine the loss value of the image detection model ; Use the loss value to adjust the network parameters of the image detection model with a preset learning rate.

Therefore, using the set similarity loss function to process the actual area and the detection area to determine the loss value of the image detection model can ensure the accuracy of the loss value, so as to use the loss value to adjust the network of the image detection model with a preset learning rate The parameters can reduce the difference between the detection area and the actual area during the training process, and improve the accuracy of the image detection model.

The embodiment of the present application also provides an image detection device, including an image acquisition module, a feature extraction module, an image generation module, an image fusion module and an image detection model, where the image acquisition module is configured to acquire a medical image to be tested; the feature extraction module is configured as Perform feature extraction on the medical image to be tested to obtain first feature maps of several dimensions; the image generation module is configured to use the first feature map of preset dimensions as a reference feature map, and use the reference feature map to generate a lesion probability map, wherein the lesion probability The map is used to represent the probability that different areas of the medical image to be tested belong to the lesion; the image fusion module is configured to fuse the lesion probability map with the first feature maps of several dimensions to obtain the final fusion feature map; the detection processing module is configured to The feature map is fused to perform detection processing, and the detection result of the lesion in the medical image to be tested is obtained.

An embodiment of the present application further provides an electronic device, including a memory and a processor coupled to each other, where the processor is configured to execute program instructions stored in the memory, so as to implement the image detection method in the first aspect.

Embodiments of the present application further provide a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the image detection method in the first aspect above is implemented.

Embodiments of the present application further provide a computer program, including computer-readable codes. When the computer-readable codes are run in an electronic device, a processor in the electronic device executes any one of the above image detection methods.

In the above solution, by performing feature extraction on the obtained medical image to be tested, first feature maps of several dimensions are obtained, and the first feature map of preset dimensions is used as a reference feature map, so as to generate a lesion probability map by using the reference feature map , and the lesion probability map is used to represent the probability that different areas of the medical image to be tested belong to the lesion, and then the lesion probability map is fused with the first feature maps of several dimensions to obtain the final fusion feature map, so the lesion probability map can be used as a global The features are fused with the first feature map, so that the final fusion feature map can enhance the specificity of the lesion, and then the detection result of the lesion in the medical image to be tested can be obtained by performing detection processing through the final fusion feature map, which can improve image detection. accuracy.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

1 is a schematic flowchart of an embodiment of an image detection method of the present application;

2 is a schematic diagram of a framework of an embodiment of an image detection model;

3 is a schematic flowchart of another embodiment of the image detection method of the present application;

4 is a schematic flowchart of an embodiment of a training image detection model;

FIG. 5 is a schematic frame diagram of an embodiment of an image detection apparatus of the present application;

6 is a schematic diagram of a framework of an embodiment of an electronic device of the present application;

FIG. 7 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application.

Detailed ways

The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

In the following description, for purposes of illustration and not limitation, specific details such as specific system structures, interfaces, techniques, etc. are set forth in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship. Also, "multiple" herein means two or more than two.

Please refer to FIG. 1 , which is a schematic flowchart of an embodiment of an image detection method of the present application. Specifically, the following steps can be included:

Step S11: Acquire the medical image to be tested.

The medical images to be tested may include CT images and nuclear magnetic resonance (Magnetic Resonance, MR) images, which are not limited herein. In an implementation scenario, the medical image to be tested may be an image obtained by scanning a lung area, a liver area, a heart area, etc., which is not limited here, and may be set according to actual application conditions. For example, the lung area can be scanned when the lungs need to be tested to screen for pneumonia; or the liver can be scanned when the liver needs to be tested to screen for changes in the liver Area scanning, etc., other application situations can be deduced by analogy, and will not be listed one by one here.

In one implementation scenario, the medical image to be tested may be a two-dimensional medical image; in another implementation scenario, the medical image to be tested may also be a two-dimensional medical image included in a three-dimensional medical image, for example, a CT scan is performed on a scanned object After obtaining the three-dimensional CT data, the medical image to be tested may be a two-dimensional medical image included in the three-dimensional CT data.

Step S12: Perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions.

In an implementation scenario, the medical image to be tested may also be preprocessed before the feature extraction, for example, at least a preset window value is used to normalize the pixel value of the medical image to be tested to a preset range, so that it can be The contrast of the medical image to be tested is enhanced, and the accuracy of the extracted first feature map is improved. Specifically, the preset window value can be set according to the scanning position. For example, when the medical image to be tested is a CT image obtained by scanning the lungs, the preset window value can be -1400 to 100 Hounsfield Unit (HU) , other parts can be set according to the actual situation, and no examples will be given here. In addition, the preset range can be set to 0 to 1, so that when the medical image to be tested is a CT image obtained by scanning the lungs, and the preset window value is -1400 to 100 Heinz units, the pixel value can be set lower than -1400 set the pixels of the -1400 to -1400, and set the pixels with a pixel value higher than 100 to 100, and finally map the pixel values in the range of -1400 to 100 to the range of 0 to 1. When the preset window value and the preset range are other values, it can be deduced by analogy, and the examples will not be exemplified here.

In an implementation scenario, in order to improve the convenience of feature extraction, an image detection model may also be pre-trained, and the image detection model may include a feature extraction sub-network, so that the feature extraction sub-network of the image detection model Feature extraction to obtain a first feature map of several dimensions. Several dimensions can be one dimension or multiple dimensions, for example, two dimensions, three dimensions, etc., which are not limited here. Specifically, the network depth of the feature extraction sub-network is set according to the actual situation, so as to obtain different dimension of the first feature map. The higher the dimension, the larger the number of channels of the corresponding first feature map and the smaller the resolution.

Please refer to FIG. 2 , which is a schematic diagram of a framework of an embodiment of an image detection model. As shown in Figure 2, the feature extraction sub-network may include multiple sequentially connected feature extraction sub-modules, and the feature extraction sub-module may perform processing tasks such as convolution, regularization, activation, and pooling. Specifically, the feature extraction sub-module may include Any one of Residual Block, Inception Block, and Dense Block is not limited here. Specifically, the pooling process may include any one of max pooling (Max Pooling), average pooling (Average Pooling), and a convolutional layer with a stride (Stride) of 2, which is not limited herein. In the process of feature extraction performed by sequentially connected feature extraction sub-modules, first feature maps from low to high dimensions can be sequentially obtained, and when the feature extraction sub-modules are executed, the number of channels is doubled and the resolution is halved. For example, the resolution of the medical image to be tested is 256*256, the number of channels of the first feature map extracted by the first feature extraction sub-module in FIG. 2 is 64, and the resolution is 128*128. The size of the map is uniformly expressed as the number of channels * resolution, that is, 64*128*128. The size of the first feature map extracted by the second feature extraction sub-module connected in sequence is 128*64*64, and the size of the third feature extraction is 128*64*64. The size of the first feature map extracted by the submodule is 256*32*32, and the size of the first feature map extracted by the fourth feature extraction submodule is 512*16*16. When the feature extraction sub-network is of other structures, it can be deduced in the same way, and no examples will be given here.

Step S13 : using the first feature map of the preset dimension as a reference feature map, and using the reference feature map to generate a lesion probability map.

The lesion probability map is used to represent the probability that different regions in the medical image to be tested belong to the lesion. Still taking the resolution of the medical image to be tested as 256*256 as an example, the reference feature map may include the first feature map with the highest dimension, that is, the first feature map with a size of 512*16*16, and each pixel point in the reference feature map It can correspond to a 16*16 area in the medical image to be tested, so the pixel value of each pixel in the lesion probability map generated by the reference feature map can represent the probability that a 16*16 area in the medical image to be tested belongs to the lesion . In addition, the reference feature map may also include a first feature map of other dimensions. For example, the reference feature map may also include a first feature map that is one dimension lower than the first feature map with the highest dimension, which is not limited herein. In other implementation scenarios, according to the specific application situation, the first feature map of other dimensions may also be selected with reference to the feature map, which is not limited herein.

In an implementation scenario, in order to improve the accuracy of the lesion probability map, a class activation map (CAM) can be generated by counting the gradient values of each pixel in the reference feature map with respect to the lesion, and the class activation map can be used as the Lesion probability map. In a specific implementation scenario, the reference feature map can be used to perform prediction processing to obtain the first probability value y ^c of the lesion contained in the medical image to be tested, so that by calculating the first probability value y ^c for each pixel of the reference feature map

The gradient value of

in,

where k represents the kth reference feature map in the reference feature maps of multiple channels, and ij represents the pixel point in the ith row and the jth column in the kth reference feature map. Specifically, an image detection model can be pre-trained, and the image detection model includes a prediction processing sub-network, so that the prediction processing sub-network can be used to perform prediction processing on the reference feature map to obtain the first probability that the medical image to be tested contains a lesion value. Please continue to refer to Figure 2. The prediction processing sub-network can include a global average pooling and a fully connected layer. Still taking the resolution of the medical image to be tested as 256*256 as an example, for the reference feature with a size of 512*16*16 The image is subjected to Global Average Pooling (GAP) processing to obtain a vector with a size of 512*1*1, and the fully connected layer is used to process the vector to obtain the first probability value that the medical image to be tested contains a lesion .

In one implementation scenario, in order to pre-screen negative data before detection and solve the problem of false positive results obtained in subsequent detection, that is, there is no lesion in the medical image to be tested, but the lesion is obtained by detecting the medical image to be tested. False positive results will interfere with clinical applications, so the reference feature map can be used for prediction processing to obtain the first probability value of the lesion contained in the medical image to be tested, and based on the first probability value, it is determined whether to use the reference feature map to generate the lesion probability Figure steps and next steps. In a specific implementation scenario, the method of using the reference feature map to perform prediction processing to obtain the first probability value may refer to the relevant description in the foregoing implementation scenario, and details are not repeated here. In another specific implementation scenario, a probability threshold can be preset, and when the first probability value is lower than the probability threshold, it can be considered that the medical image to be tested does not contain lesions, and the use of reference feature maps to generate lesions can no longer be performed. The steps of the probability map and subsequent steps can greatly reduce the possibility of obtaining false positive results by still performing image detection on the medical image to be tested in this case. The probability threshold can be set according to the actual application. For example, it can be set to a value lower than 20%, 30%, etc., so that the medical images to be tested that have a high probability of not containing lesions are not subjected to subsequent detection, so as to improve the detection efficiency. , and reduce the likelihood of false positive results.

Step S14 : fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fusion feature map.

Specifically, in the process of fusion, the lesion probability map and the first feature maps of several dimensions may be fused according to the order of dimensions from high to low to obtain the final fusion feature map. In one implementation scenario, the reference feature map can be encoded by using the lesion probability map to obtain a second feature map, and the second feature map can be fused with the first feature maps of several dimensions to obtain the final fused feature map, thereby The lesion probability map is used as a global feature to participate in the feature map fusion, so that the final fusion feature map can enhance the specificity of the lesion, which can help improve the accuracy of subsequent image detection. Still taking the resolution of the medical image to be tested as 256*256 as an example, the size of the reference feature map is 512*16*16, corresponding to each reference feature map of 512 channels, and the corresponding lesion probability map is obtained. The size of the probability map is 512*16*16, and then the lesion probability map of size 512*16*16, the reference feature map of size 512*16*16, and the first feature map of size 256*32*32 can be combined , the first feature map with a size of 128*64*64, and the first feature map with a size of 64*128*128 are fused to obtain the final fusion feature map with a channel number of 1.

In a specific implementation scenario, in order to reduce the computational complexity of the encoding process, the pixel value of the first pixel in the lesion probability map can be directly compared with the pixel value of the second pixel corresponding to the first pixel in the reference feature map. Multiply to obtain the pixel value of the corresponding pixel of the second feature map. Still taking the resolution of the medical image to be tested as 256*256 as an example, through the above processing, a reference feature map with a size of 512*16*16 and a corresponding lesion probability map with a size of 512*16*16 can be obtained, so For each channel, the pixel value of the first pixel in the lesion probability map can be multiplied by the pixel value of the second pixel corresponding to the first pixel in the reference feature map of the corresponding channel to obtain the second pixel of the corresponding channel. For the pixel value of the corresponding pixel of the feature map, by performing the same processing on 512 channels, a second feature map with a size of 512*16*16 can be obtained. When the medical image to be tested is an image of other resolutions, or, when the reference feature map is an image of other sizes, it can be deduced by analogy, and no examples will be given here.

In another specific implementation scenario, in order to improve the accuracy and richness of the final fused image, the second feature map and the first feature map of each dimension sorted according to the above order can be arranged in the order of dimensions from high to low. Fusion is performed to obtain the final fusion feature map, which can be beneficial to fully fuse context information. Specifically, the reference feature map and the first low-dimensional feature map may be fused to obtain a first fused feature map with the same dimension as the first low-dimensional feature map, and the first low-dimensional feature map is a first low-dimensional feature map lower than the reference feature map. One-dimensional first feature map, and fuse the second feature map with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map, and then repeat the execution of the second fused feature map and the first fused feature map. Two low-dimensional feature maps are fused to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, wherein the second low-dimensional feature map is a ratio of The first feature map that is one dimension lower than the current second fusion feature map, and the second fusion feature map obtained by final fusion can be used as the final fusion feature map. Still taking the resolution of the medical image to be tested as 256*256 as an example, the reference feature map with a size of 512*16*16 and its corresponding first low-dimensional feature map, that is, the first feature with a size of 256*32*32 During the fusion process, the number of channels of the reference feature map with a size of 512*16*16 can be halved and the resolution doubled, so as to adjust its size to be the same as the corresponding first low-dimensional feature map, and then Combine the adjusted reference feature map and the first low-dimensional feature map with a size of 256*32*32 into a feature map with a size of 512*32*32. In order to facilitate subsequent fusion, the number of channels can be halved to obtain The first fused feature map with the same size as the first low-dimensional feature map, that is, the first fused feature map with a size of 256*32*32. Then fuse the second feature map with a size of 512*16*16 and the first fusion feature map. Similarly, during the fusion process, you can first halve the number of channels of the second feature map with a size of 512*16*16, And multiply the resolution to adjust its size to be the same as the first fused feature map, and then merge the adjusted second feature map with the first fused feature map of size 256*32*32 into a size of 512*32* 32 feature map, in order to facilitate subsequent fusion, the number of channels can be halved to obtain a second fused feature map with the same size as the first fused feature map, that is, a second fused feature map with a size of 256*32*32. Then perform the step of fusing the second fusion feature map with a size of 256*32*32 and the corresponding second low-dimensional feature map (ie, the second feature map with a size of 128*64*64). Similarly, the fusion process , you can first halve the number of channels of the second fusion feature map with a size of 256*32*32 and multiply the resolution, so as to adjust its size to be consistent with the corresponding second low-dimensional feature map, and then adjust the adjusted The second fusion feature map and the corresponding second low-dimensional feature map (that is, the second feature map with a size of 128*64*64) are combined into a feature map with a size of 256*64*64. The number of channels is halved, and a new second fusion feature map with the same size as the corresponding second low-dimensional feature map is obtained, that is, a second fusion feature map with a size of 128*64*64. Then perform the step of fusing the second fusion feature map with a size of 128*64*64 and the corresponding second low-dimensional feature map (ie, the second feature map with a size of 64*128*128). Similarly, the fusion process , you can first halve the number of channels of the second fusion feature map with a size of 128*64*64 and multiply the resolution, so as to adjust its size to be consistent with the corresponding second low-dimensional feature map, and then adjust the adjusted The second fusion feature map and the corresponding second low-dimensional feature map (that is, the second feature map with a size of 64*128*128) are combined into a feature map with a size of 128*128*128. In order to facilitate subsequent processing, the same The number of channels is halved, and a new second fusion feature map with the same size as the corresponding second low-dimensional feature map is obtained, that is, a second fusion feature map with a size of 64*128*128. The feature maps are all fused, so the second fused feature map with a size of 64*128*128 obtained by the final fusion can be used as the final fused feature map. Other situations can be deduced by analogy, and no examples are given here.

In another specific implementation scenario, in order to improve the efficiency of fusion processing, an image detection model can be pre-trained, and the image detection model includes a fusion processing sub-network, so that the fusion processing sub-network of the image detection model is used to combine the lesion probability map with the The first feature maps of several dimensions are fused to obtain the final fused feature map. Specifically, please continue to refer to FIG. 2. As shown in FIG. 2, the fusion processing sub-network includes a plurality of sequentially connected fusion processing sub-modules, and is configured to perform fusion of the lesion probability map and the first feature maps of several dimensions, Steps to get the final fused feature map. Each fusion processing sub-module can perform processing operations such as upsampling, convolution, regularization, activation, merging, convolution, regularization, and activation. Taking the fusion process of a higher-dimensional feature map with a size of 512*16*16 and a corresponding low-dimensional feature map with a size of 256*32*32 in the aforementioned implementation scenario as an example, the upsampling process is used to Resolution doubling of higher-dimensional feature maps of size 512*16*16, resulting in feature maps of size 512*32*32; the first convolution process is used to double the resolution during fusion. The number of channels of the dimensional feature map (that is, the feature map with a size of 512*32*32) is halved to obtain a feature map with the same size as the corresponding low-dimensional feature map (that is, the size is adjusted to 256*32*32); In the fusion process, the adjusted higher-dimensional feature map and its corresponding low-dimensional feature map are combined to double the number of channels, that is, the size of the combined feature map is 512*32*32, and the second convolution process In the fusion process, the feature map obtained by doubling the number of channels obtained after merging is halved again, so that the size of the fusion feature map obtained by this fusion is the same as the size of the corresponding low-dimensional feature map, that is, the size is 256 *32*32 fused feature map. For details, reference may be made to the relevant steps in the foregoing implementation scenarios, and details are not repeated here.

Step S15 : performing detection processing on the final fusion feature map to obtain a detection result about the lesion in the medical image to be tested.

In an implementation scenario, for the convenience of the doctor to view, the detection result may include the detection area of the lesion in the image to be measured. Specifically, a line of a preset color and a preset line type may be used to represent the detection area, which is not limited herein. In another implementation scenario, the medical image to be tested is a two-dimensional medical image included in the three-dimensional medical image, so the detection area detected in the two-dimensional medical image can be used to obtain the detection area of the lesion in the three-dimensional medical image, for example, The detection area detected in the 2D medical image can be fused in a 3D space such as stacking, so as to obtain the detection area of the lesion in the 3D medical image, which can be set according to the actual application, which is not limited here.

In another implementation scenario, in order to improve the efficiency of detection processing, an image detection model can be pre-trained, and the image detection model can include a fusion processing sub-network, so that the fusion processing sub-network of the image detection model can be used to fuse the final fusion feature map. A detection process is performed to obtain a detection result about the lesion in the medical image to be tested. Specifically, please continue to refer to FIG. 2 , the fusion processing sub-network may include, in addition to a plurality of fusion processing sub-modules connected in sequence, an activation processing sub-network connected to the last of the sequentially connected plurality of fusion processing sub-modules. module, the activation processing sub-module is configured to convolve and activate the final fusion feature image to obtain a feature map with a channel number of 1, and normalize the feature map to obtain the detection result of the lesion. Specifically, the activation processing sub-module may include a sequentially connected convolution layer and an activation layer, and the activation layer may adopt a sigmoid activation function, which may be set according to actual application conditions, which is not limited here.

In yet another implementation scenario, in order to provide clinical reference, organ detection can also be performed on the medical image to be tested to obtain the organ region in the medical image to be tested, so that the proportion of the lesion occupied by the detection region of the lesion in the organ region can be obtained, and then It can provide doctors with clinical reference information and improve user experience. For example, lung detection can be performed on the medical image to be tested to obtain the lobe area of the lung in the medical image to be tested, so that the proportion of the lesion occupied by the detected area of the lesion in the lobe area of the lung can be obtained. Other application scenarios can be deduced by analogy, which is not limited here. Specifically, in order to improve the efficiency of organ detection, an organ detection model can also be pre-trained, so that the organ detection model can be used to perform organ detection on the medical image to be tested to obtain the organ region in the medical image to be tested. Specifically, the organ detection model may be based on U-net, fully convolutional networks (Fully Convolutional Networks, FCN), etc., which are not limited here.

Please refer to FIG. 3 , which is a schematic flowchart of another embodiment of the image detection method of the present application. Specifically, the following steps can be included:

Step S31: Acquire the medical image to be tested.

For details, refer to the relevant steps in the foregoing embodiments.

Step S32: Perform feature extraction on the medical image to be tested to obtain first feature maps of several dimensions.

For details, refer to the relevant steps in the foregoing embodiments.

Step S33 : using the first feature map of the preset dimension as a reference feature map, and performing prediction processing by using the reference feature map to obtain a first probability value that the medical image to be tested contains a lesion.

Specifically, an image detection model can be pre-trained, and the image detection model includes a prediction processing sub-network, so that the prediction processing sub-network can be used to perform prediction processing on the reference feature map to obtain the first probability that the medical image to be tested contains a lesion value. For details, reference may be made to the relevant steps in the foregoing embodiments, which will not be repeated here.

Step S34: Based on the first probability value, determine whether to perform the step of generating the probability map of the lesion by using the reference feature map and the subsequent steps, if yes, perform step S35, otherwise, perform step S38.

In an implementation scenario, when the first probability value satisfies the first preset condition, it may be determined to perform the step of generating the lesion probability map by using the reference feature map and the subsequent steps. Specifically, the first preset condition may include that the first probability value is greater than or equal to the first probability threshold, and the first probability threshold may be set according to practical applications, for example, may be set to 15%, 20%, 25%, etc., here Not limited.

In another implementation scenario, the medical image to be tested is a two-dimensional medical image included in the three-dimensional medical image, so the first probability values of the two-dimensional medical image can be sorted in descending order, and the pre-set probability value can be selected The number of first probability values, the preset number can be set according to the actual situation, for example, can be set to 5, 6, etc., which is not limited here. Preset processing is performed on a preset number of first probability values, for example, an average operation is performed on a preset number of first probability values to obtain a second probability value, and when the second probability value satisfies the second preset condition, The step of generating the lesion probability map using the reference feature map and subsequent steps may be determined to be performed. Specifically, the second preset condition may include that the second probability value is greater than or equal to the second probability threshold, and the second probability threshold may be set according to practical applications, for example, may be set to 15%, 20%, 25%, etc. Do limit.

Step S35 : generating a lesion probability map using the reference feature map.

For details, refer to the relevant steps in the foregoing embodiments.

Step S36 : fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fusion feature map.

For details, refer to the relevant steps in the foregoing embodiments.

Step S37: Perform detection processing on the final fusion feature map to obtain a detection result about the lesion in the medical image to be tested.

For details, refer to the relevant steps in the foregoing embodiments.

Step S38: Prompt that the medical image to be tested does not contain a lesion.

When it is determined based on the first probability value that the step of generating the lesion probability map using the reference feature map and subsequent steps are not to be performed, it may be determined that the medical image to be tested does not contain a lesion. In addition, it is also possible to prompt the medical staff that the medical image to be tested does not contain a lesion by means of text, image, voice, etc., which is not limited herein.

It can be seen that, by using the reference feature map for prediction processing, the first probability value of the lesion contained in the medical image to be tested is obtained, and based on the first probability value, it is determined whether to perform the step of using the reference feature map to generate the lesion probability map and subsequent steps. , which can avoid false positive detection results when the medical image to be tested does not contain lesions, which can further improve the accuracy of image detection, and because the negative data can be pre-screened before detection, it can improve image detection. s efficiency.

Please refer to FIG. 4 , which is a schematic flowchart of an embodiment of training an image detection model. Specifically, the image detection model may be trained before using the feature extraction sub-network of the image detection model to perform feature extraction on the medical image to be tested. For the network structure of the image detection model, reference may be made to the foregoing embodiments, which will not be repeated here. Specifically, the training steps may include:

Step S41: Obtain a sample medical image, wherein the sample medical image includes the actual area of the lesion.

The sample medical images may include CT images and MR images, which are not limited herein. Specifically, the sample medical image may be an image obtained by scanning a lung area, a liver area, a heart area, etc., which is not limited here, and may be specifically set according to actual application conditions. In an implementation scenario, the sample medical image may be a 2D medical image included in the 3D medical image. For example, if the 3D CT data is obtained by performing a CT scan on a scanned object, the sample medical image may be a 2D medical image included in the 3D CT data. dimensional medical images.

In one implementation scenario, in order to improve the diversity of samples, data enhancement can also be performed on the sample medical image; in another implementation scenario, in order to improve the contrast of the sample medical image, the pixels of the sample medical image can also be enhanced by a preset window value. Values are normalized to within a preset range. For the specific setting method of the preset window value and the preset range, reference may be made to the relevant steps in the foregoing embodiments, and details are not described herein again.

Step S42 : using the feature extraction sub-network to perform feature extraction on the sample medical image to obtain a first sample feature map of several dimensions.

For details, refer to the relevant steps in the foregoing embodiments.

Step S43 : using the first sample feature map of the preset dimension as the reference sample feature map, and using the reference sample feature map to generate a lesion sample probability map.

The lesion sample probability map is used to represent the probability that different regions in the sample medical image belong to the lesion. For details of the acquisition method of the probability map of the lesion sample, reference may be made to the steps of acquiring the probability map of the lesion in the foregoing embodiment, which will not be repeated here.

Step S44 : using the fusion processing sub-network to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain a final fused sample feature map.

For details, reference may be made to the relevant steps in the foregoing embodiments, which will not be repeated here.

Step S45 : use the fusion processing sub-network to perform detection processing on the final fusion sample feature map to obtain a detection area related to the lesion in the sample medical image.

Step S46: Using the difference between the actual area and the detection area, adjust the network parameters of the image detection model.

In an implementation scenario, a set similarity loss function (Dice loss) can be sampled to process the actual area and the detection area to determine the loss value of the image detection model, so as to use the loss value with a preset learning rate (for example, 3e-4 ) to adjust the network parameters of the image detection model. In another implementation scenario, a cross-entropy loss function (CE loss) can also be used to process the actual area and the detection area to determine the loss value of the image detection model, so as to use the loss value to use a preset learning rate (for example, 3e- 4) Adjust the network parameters of the image detection model. This is not limited. In yet another implementation scenario, please refer to FIG. 2 in combination, the image detection model further includes a prediction processing sub-network, and the prediction processing sub-network is used to perform prediction processing on the feature map of the reference sample to obtain the predicted probability of the lesion included in the feature map of the reference sample, In the training process, the prediction probability can also be processed by the binary cross-entropy loss function to determine the classification loss value of the image detection model, and the loss value and classification loss of the image detection model determined by processing the actual area and the detection area The value is weighted to obtain the weighted loss value of the image detection model, and then the network parameters of the image detection model are adjusted by using the weighted loss value.

In an implementation scenario, a training end condition may also be preset, and when the preset training end condition is satisfied, the training of the image detection model may be ended. Specifically, the training end condition may include any one of: the loss value is less than a preset loss threshold, and the number of training times reaches a preset number of times threshold, which is not limited herein. Specifically, the preset loss threshold and the preset number of times threshold may be set according to actual conditions. For example, the preset number of times threshold may be set to 1000 times, 2000 times, and so on. This is not limited.

In an implementation scenario, methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), Mini-Batch Gradient Descent (MBGD), etc. can be used to utilize the loss value pair The network parameters of the image detection model are adjusted. Among them, batch gradient descent means that all samples are used to update parameters at each iteration; stochastic gradient descent means that one sample is used to update parameters at each iteration; mini-batch gradient descent means that at each iteration When , a batch of samples is used to update the parameters, which will not be repeated here.

It can be seen that by obtaining the sample medical image, and the sample medical image contains the actual area of the lesion, and using the feature extraction sub-network to perform feature extraction on the sample medical image, a first sample feature map of several dimensions is obtained, so as to predict The first sample feature map of the dimension is set as the reference sample feature map, and the reference sample feature map is used to generate the lesion sample probability map, and the lesion sample probability map is used to represent the probability that different regions in the sample medical image belong to the lesion, and then the lesion is classified as a lesion. The probability map is fused with the first sample feature map of several dimensions to obtain the final fused sample feature map, so as to perform detection processing on the final fused sample feature map to obtain the detection area of the lesion in the sample medical image, and use the actual area and detection. The difference between regions can be adjusted by adjusting the network parameters of the image detection model. Therefore, in the training process of the image detection model, the probability map of the lesion sample can be used as a global feature to couple with the decoding process of image detection, so that the final fusion sample feature map can be strengthened. The specificity of the lesion can enhance the sensitivity of the image detection model to the lesion, which can help improve the training speed of the model.

Please refer to FIG. 5 , which is a schematic diagram of a framework of an embodiment of an image detection apparatus 50 of the present application. The image detection device 50 includes an image acquisition module 51, a feature extraction module 52, an image generation module 53, an image fusion module 54 and a detection processing module 55. The image acquisition module 51 is configured to acquire a medical image to be tested; the feature extraction module 52 is configured to be tested. Perform feature extraction on the medical image to obtain a first feature map of several dimensions; the image generation module 53 is configured to use the first feature map of a preset dimension as a reference feature map, and use the reference feature map to generate a lesion probability map, wherein the lesion probability map It is configured to represent the probability that different areas of the medical image to be tested belong to the lesion; the image fusion module 54 is configured to fuse the lesion probability map with the first feature maps of several dimensions to obtain the final fusion feature map; the detection processing module 55 is used for Finally, the feature map is fused for detection processing, and the detection result of the lesion in the medical image to be tested is obtained.

In some embodiments, the image detection apparatus 50 further includes a prediction processing module, the prediction processing module is configured to perform prediction processing using the reference feature map to obtain a first probability value that the medical image to be tested contains a lesion; the image detection apparatus 50 further includes An execution determination module is configured to determine, based on the first probability value, whether to execute the step of generating a lesion probability map using the reference feature map and subsequent steps.

It can be seen that, by using the reference feature map for prediction processing, the first probability value of the lesion contained in the medical image to be tested is obtained, and based on the first probability value, it is determined whether to perform the step of using the reference feature map to generate the lesion probability map and subsequent steps, In this way, false positive detection results can be avoided when the medical image to be tested does not contain lesions, which can further improve the accuracy of image detection, and because the negative data can be pre-screened before detection, the accuracy of image detection can be improved. efficiency.

The execution determination module is specifically configured to, when the first probability value satisfies the first preset condition, execute the step of generating the lesion probability map by using the reference feature map and the subsequent steps;

Or, when the medical image to be tested is a two-dimensional medical image included in the three-dimensional medical image, the execution determination module further includes a probability selection sub-module; the probability selection sub-module is configured to select the first probability value of the two-dimensional medical image according to Sorting from large to small, and selecting the first preset number of first probability values; the execution determination module further includes a probability processing sub-module, and the probability processing sub-module is configured to perform preset processing on a preset number of first probability values to obtain the second probability value; the execution determination module further includes a determination execution sub-module, the determination sub-module is configured to execute the step of using the reference feature map to generate the lesion probability map and subsequent steps when the second probability value satisfies the second preset condition .

It can be seen that when the first probability value satisfies the first preset condition, the step of using the reference feature map to generate the lesion probability map and subsequent steps are performed, or, when the medical image to be tested is a two-dimensional medical image included in a three-dimensional medical image , sorting the first probability values of the two-dimensional medical images in descending order, selecting the first preset number of first probability values, and performing preset processing on the preset number of first probability values to obtain the second probability value, so that when the second probability value satisfies the second preset condition, the step of using the reference feature map to generate the probability map of the lesion and the subsequent steps are performed, which is beneficial to pre-screening negative data before detection, thereby improving the image quality. Detection accuracy and efficiency.

In some embodiments, the first preset condition includes: the first probability value is greater than or equal to the first probability threshold; the second preset condition includes: the second probability value is greater than or equal to the second probability threshold; the preset processing is an average operation .

It can be seen that by setting the first preset condition as the first probability value is greater than or equal to the first probability threshold, and by setting the second preset condition as the second probability value greater than or equal to the second probability threshold, the preset processing setting is an average operation, so that the calculation amount of the second probability value can be reduced, and the second probability value can accurately reflect the possibility that the three-dimensional medical image contains lesions. Therefore, when the first probability value is greater than or equal to the first probability threshold, Execute the step of using the reference feature map to generate the case probability map and subsequent steps, and when the second probability value is greater than or equal to the second probability threshold, execute the step of using the reference feature map to generate the case probability map and subsequent steps, so it can be beneficial to the detection of Negative data is pre-screened to improve the accuracy and efficiency of image detection.

In some embodiments, the execution determination module is further configured to determine that the medical image to be tested does not contain a lesion when the first probability value does not satisfy the first preset condition or the second probability value does not satisfy the second preset condition.

It can be seen that when the first probability value does not meet the first preset condition or the second probability value does not meet the second preset condition, it is determined that the medical image to be tested does not contain a lesion, so that the user can perceive the medical image to be tested in time. negative test results, which can help improve the user experience.

In some embodiments, the image generation module 53 is specifically configured to count the gradient values of each pixel in the reference feature map with respect to the lesion, generate a class activation map, and use the class activation map as the lesion probability map.

It can be seen that by counting the gradient values of each pixel in the reference feature map with respect to the lesion, the class activation map is generated as the lesion probability map, which can improve the accuracy of the lesion probability map, which can help improve the accuracy of subsequent image detection. .

In some embodiments, the image fusion module 54 includes an encoding processing submodule, and the encoding processing submodule is configured to perform encoding processing on the reference feature map by using the lesion probability map to obtain the second feature map; the image fusion module 54 includes a fusion processing submodule, The fusion processing sub-module is configured to fuse the second feature map with the first feature maps of several dimensions to obtain a final fusion feature map.

It can be seen that by using the lesion probability map to encode the reference feature map, the second feature map is obtained, and the second feature map is fused with the first feature maps of several dimensions to obtain the final fusion feature map, so it can be The lesion probability map is used as a global feature to participate in the feature map fusion, so that the final fusion feature map can enhance the specificity of the lesion, which can help improve the accuracy of subsequent image detection.

In some embodiments, the encoding processing sub-module is specifically configured to multiply the pixel value of the first pixel point in the lesion probability map and the pixel value of the second pixel point corresponding to the first pixel point in the reference feature map to obtain the second pixel value. The pixel value of the corresponding pixel of the feature map.

It can be seen that by multiplying the pixel value of the first pixel in the lesion probability map and the pixel value of the second pixel corresponding to the first pixel in the reference feature map, the pixel corresponding to the pixel in the second feature map is obtained. value, so as to realize the encoding processing of the reference feature map from the lesion probability map, which can help to reduce the amount of calculation.

In some embodiments, the fusion processing sub-module is specifically configured to fuse the second feature map with the first feature map of each dimension in order in order of dimensions from high to bottom to obtain a final fused feature map.

It can be seen that by merging the second feature map with the first feature map of each dimension in the order of dimensions from high to low to obtain the final fusion feature map, it can be beneficial to perform feature map fusion dimension by dimension, so as to be able to It is beneficial to fully fuse context information, improve the accuracy and feature richness of the final fusion feature map, and further improve the accuracy of subsequent image detection.

In some embodiments, the reference feature map is the first feature map with the highest dimension; the fusion processing sub-module includes a first fusion unit, and the first fusion unit is configured to fuse the reference feature map and the first low-dimensional feature map to obtain the same The first fusion feature map with the same dimension of the first low-dimensional feature map, wherein the first low-dimensional feature map is a first feature map with one dimension lower than the reference feature map;

The fusion processing submodule includes a second fusion unit, and the second fusion unit is configured to fuse the second feature map with the first fusion feature map to obtain a second fusion feature map with the same dimension as the first fusion feature map;

The fusion processing sub-module includes a third fusion unit, and the third fusion unit is configured to repeatedly perform the fusion of the second fusion feature map and the second low-dimensional feature map to obtain a new second low-dimensional feature map with the same dimension as the second low-dimensional feature map. Fusing the feature maps until the first feature maps of several dimensions are fused; wherein, the second low-dimensional feature map is the first feature map one dimension lower than the current second fused feature map;

The fusion processing sub-module includes a final fusion unit, and the final fusion unit is configured to use the second fusion feature map obtained by final fusion as the final fusion feature map.

It can be seen that by fusing the reference feature map with the first low-dimensional feature map, the first fusion feature map with the same dimension as the first low-dimensional feature map is obtained, and the first low-dimensional feature map is lower than the reference feature map. One-dimensional first feature map, and fuse the second feature map with the first fused feature map to obtain a second fused feature map with the same dimension as the first fused feature map. The second low-dimensional feature map is fused to obtain a new second fused feature map with the same dimension as the second low-dimensional feature map, until the first feature maps of several dimensions are fused, and the second low-dimensional feature map is a ratio of The current second fusion feature map is one dimension lower than the first feature map, and the second fusion feature map obtained by final fusion is used as the final fusion feature map, and the lesion probability map can be used as a global feature to couple with the decoding process of image detection, The final fusion feature map can enhance the specificity of the lesion, and can fully integrate the context information of the feature map, improve the accuracy and feature richness of the final fusion feature map, and further improve the accuracy of subsequent image detection.

In some embodiments, the detection result includes the detection area of the lesion in the medical image to be tested; the image detection apparatus 50 further includes an organ detection module, and the organ detection module is configured to perform organ detection on the medical image to be tested to obtain the organ in the medical image to be tested. area; the image detection apparatus 50 further includes a proportion acquisition module, which is configured to acquire the proportion of the lesion in the organ area occupied by the detection area of the lesion.

It can be seen that, by performing organ detection on the medical image to be tested, the organ region in the medical image to be tested can be obtained, and the proportion of the lesion occupied by the detection region of the lesion in the organ region can be obtained. Clinical reference information to improve user experience.

In some embodiments, the image detection apparatus 50 further includes a preprocessing module, and the preprocessing module is configured to preprocess the medical image to be tested, wherein the preprocessing operation at least includes: using a preset window value to convert pixels of the medical image to be tested. Values are normalized to within a preset range.

It can be seen that, before the feature extraction is performed on the medical image to be tested, the medical image to be tested is preprocessed, and the preprocessing operation at least includes: using a preset window value to normalize the pixel value of the medical image to be tested to a preset range It can help to enhance the contrast of the medical image to be tested, and thus can help to improve the accuracy of the subsequently extracted first feature map.

In some embodiments, the feature extraction module 52 is specifically configured to use the feature extraction sub-network of the image detection model to perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions; the image fusion module 54 is specifically configured to use image detection The fusion processing sub-network of the model fuses the lesion probability map with the first feature maps of several dimensions to obtain the final fusion feature map; the detection processing module 55 is specifically configured to use the fusion processing sub-network of the image detection model to perform the final fusion feature map. The detection process is performed to obtain a detection result about the lesion in the medical image to be tested.

It can be seen that by using the feature extraction sub-network of the image detection model to extract the features of the medical image to be tested, the first feature maps of several dimensions are obtained, and the fusion processing sub-network of the image detection model is used to combine the lesion probability map with several dimensions. The first feature map is fused to obtain the final fused feature map, and the fusion processing sub-network of the image detection model is used to detect and process the final fused feature map, so as to obtain the detection result of the lesion in the medical image to be tested, so as to execute the image detection model. Feature extraction, fusion processing, and image detection tasks, which can help improve the efficiency of image detection.

In some embodiments, the image detection apparatus 50 includes a sample image acquisition module, and the sample image acquisition module is configured to acquire a sample medical image, wherein the sample medical image includes the actual area of the lesion; the image detection apparatus 50 includes a sample feature extraction module, and the sample The feature extraction module is configured to use the feature extraction sub-network to perform feature extraction on the sample medical image to obtain a first sample feature map of several dimensions; the image detection device 50 includes a probability image generation module, and the probability image generation module is configured to preset dimensions. The first sample feature map is used as a reference sample feature map, and the reference sample feature map is used to generate a lesion sample probability map, wherein the lesion sample probability map is used to represent the probability that different regions in the sample medical image belong to the lesion; the image detection device 50 includes: A sample image fusion module, the sample image fusion module is configured to use a fusion processing sub-network to fuse the lesion sample probability map with the first sample feature maps of several dimensions to obtain a final fused sample feature map; the image detection device 50 includes a sample detection process. module, the sample detection processing module is configured to use the fusion processing sub-network to perform detection processing on the final fusion sample feature map to obtain the detection area about the lesion in the sample medical image; the image detection device 50 includes a training adjustment module, and the training adjustment module is configured to use the actual The difference between the region and the detection region, adjust the network parameters of the image detection model.

In some embodiments, the training adjustment module includes a loss determination submodule, and the loss determination submodule is configured to process the actual area and the detection area by using the set similarity loss function to determine the loss value of the image detection model; the training adjustment module includes parameter adjustment The sub-module, the parameter adjustment sub-module is configured to use the loss value to adjust the network parameters of the image detection model with a preset learning rate.

It can be seen that using the set similarity loss function to process the actual area and the detection area to determine the loss value of the image detection model can ensure the accuracy of the loss value, so that the loss value pair is used to adjust the image detection model with a preset learning rate The network parameters can reduce the difference between the detection area and the actual area during the training process, and improve the accuracy of the image detection model.

Please refer to FIG. 6 , which is a schematic diagram of a framework of an embodiment of an electronic device 60 of the present application. The electronic device 60 includes a memory 61 and a processor 62 coupled to each other, and the processor 62 is configured to execute program instructions stored in the memory 61 to implement the steps of any of the image detection method embodiments described above. In a specific implementation scenario, the electronic device 60 may include, but is not limited to, a microcomputer and a server. In addition, the electronic device 60 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.

Specifically, the processor 62 is configured to control itself and the memory 61 to implement the steps of any of the image detection method embodiments described above. The processor 62 may also be referred to as a central processing unit (Central Processing Unit, CPU). The processor 62 may be an integrated circuit chip with signal processing capability. The processor 62 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 62 may be jointly implemented by an integrated circuit chip.

The above solution can improve the accuracy of image detection.

Please refer to FIG. 7 , which is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 70 of the present application. The computer-readable storage medium 70 stores program instructions 701 that can be executed by the processor, and the program instructions 701 are used to implement the steps of any of the above image detection method embodiments.

The above solution can improve the accuracy of image detection.

In the several embodiments provided in this application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other divisions. For example, units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed over network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Industrial Applicability

Embodiments of the present application disclose an image detection method, device, device, computer-readable storage medium, and computer program, wherein the image detection method includes: acquiring a medical image to be tested; extracting features from the medical image to be tested to obtain several dimensions The first feature map of the first feature map; the first feature map of the preset dimension is used as the reference feature map, and the reference feature map is used to generate a lesion probability map, wherein the lesion probability map is used to indicate the probability of different regions in the medical image to be tested belong to the lesion; The lesion probability map and the first feature maps of several dimensions are fused to obtain a final fused feature map; the final fused feature map is detected and processed to obtain a detection result of the lesion in the medical image to be tested. The above solution can improve the accuracy of image detection.

Claims

An image detection method, comprising:

Obtain the medical image to be tested;

Perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions;

Taking the first feature map of a preset dimension as a reference feature map, the reference feature map is used to generate a lesion probability map, wherein the lesion probability map is used to indicate that different regions in the medical image to be tested belong to the lesion. probability;

Fusing the lesion probability map with the first feature maps of the several dimensions to obtain a final fusion feature map;

A detection process is performed on the final fusion feature map to obtain a detection result about the lesion in the medical image to be tested.
The method according to claim 1, wherein before generating the lesion probability map using the reference feature map, the method further comprises:

Perform prediction processing by using the reference feature map to obtain a first probability value that the medical image to be tested contains the lesion;

Based on the first probability value, it is determined whether to perform the step of generating a lesion probability map using the reference feature map and subsequent steps.
The method according to claim 2, wherein, based on the first probability value, determining whether to perform the step of generating a lesion probability map by using the reference feature map and subsequent steps, comprising:

If the first probability value satisfies the first preset condition, the step of generating a lesion probability map by using the reference feature map and the subsequent steps are performed; or

In the case where the medical image to be tested is a two-dimensional medical image included in a three-dimensional medical image, determining whether to perform the step of generating a lesion probability map by using the reference feature map based on the first probability value; and Next steps, including:

Sort the first probability values corresponding to the two-dimensional medical images in descending order, and select the first preset number of first probability values;

performing preset processing on the preset number of the first probability values to obtain a second probability value;

If the second probability value satisfies the second preset condition, the step of generating a lesion probability map by using the reference feature map and subsequent steps are performed.
The method according to claim 3, wherein the first preset condition includes: the first probability value is greater than or equal to a first probability threshold; the second preset condition includes: the second probability value is greater than or equal to or equal to the second probability threshold; the preset processing is an average operation;

And/or, the method further includes:

If the first probability value does not meet the first preset condition or the second probability value does not meet the second preset condition, it is determined that the medical image to be tested does not contain the lesion.
The method according to any one of claims 1 to 4, wherein the generating a lesion probability map using the reference feature map comprises:

The gradient value of each pixel in the reference feature map with respect to the lesion is counted, a class activation map is generated, and the class activation map is used as the lesion probability map.
The method according to any one of claims 1 to 5, wherein the fusion of the lesion probability map and the first feature maps of the several dimensions to obtain a final fusion feature map comprises:

Encoding the reference feature map by using the lesion probability map to obtain a second feature map;

The second feature map is fused with the first feature maps of several dimensions to obtain a final fused feature map.
The method according to claim 6, wherein the encoding process is performed on the reference feature map using the lesion probability map to obtain a second feature map, comprising:

Multiplying the pixel value of the first pixel in the lesion probability map and the pixel value of the second pixel corresponding to the first pixel in the reference feature map to obtain the corresponding pixel of the second feature map the pixel value of the point;

And/or, merging the second feature map with the first feature maps of the several dimensions to obtain a final fusion feature map, including:

According to the order of dimensions from high to bottom, the second feature map is fused with the first feature map of each dimension sorted according to the order to obtain a final fused feature map.
The method according to claim 7, wherein the reference feature map is the first feature map with the highest dimension; the second feature map and the second feature map are sorted according to the order according to the dimension from high to bottom The first feature map of each dimension of is fused to obtain the final fused feature map, including:

The reference feature map and the first low-dimensional feature map are fused to obtain a first fusion feature map with the same dimension as the first low-dimensional feature map, wherein the first low-dimensional feature map is larger than the first low-dimensional feature map. the first feature map that is one dimension lower than the reference feature map;

Fusing the second feature map with the first fusion feature map to obtain a second fusion feature map with the same dimension as the first fusion feature map;

Repeatedly performing the fusion of the second fusion feature map and the second low-dimensional feature map to obtain a new second fusion feature map with the same dimension as the second low-dimensional feature map, until all the dimensions of the several dimensions are fused. The fusion of the first feature map is completed, wherein the second low-dimensional feature map is the first feature map that is one dimension lower than the current second fusion feature map;

The second fusion feature map obtained by final fusion is used as the final fusion feature map.
The method according to any one of claims 1 to 8, wherein the detection result includes the detection area of the lesion in the medical image to be tested, and the method further comprises:

performing organ detection on the medical image to be tested to obtain an organ region in the medical image to be tested;

obtaining the proportion of the lesions in the organ region occupied by the detection area of the lesion;

And/or, before the feature extraction is performed on the medical image to be tested to obtain a first feature map of several dimensions, the method further includes:

The medical image to be tested is preprocessed, wherein the operation of the preprocessing at least includes: normalizing the pixel value of the medical image to be tested to within a preset range by using a preset window value.
The method according to any one of claims 1 to 9, wherein the feature extraction is performed on the medical image to be tested to obtain a first feature map of several dimensions, including:

Use the feature extraction sub-network of the image detection model to perform feature extraction on the medical image to be tested, to obtain the first feature map of the several dimensions;

The said lesion probability map is fused with the first feature maps of the several dimensions to obtain a final fusion feature map, including:

Using the fusion processing sub-network of the image detection model to fuse the lesion probability map with the first feature maps of several dimensions to obtain a final fusion feature map;

The detection process is performed on the final fusion feature map to obtain a detection result about the lesion in the medical image to be tested, including:

The fusion processing sub-network of the image detection model is used to perform detection processing on the final fusion feature map, so as to obtain the detection result of the lesion in the medical image to be tested.
The method according to claim 10, wherein, before the feature extraction sub-network using the image detection model performs feature extraction on the medical image to be tested, and obtains the first feature maps of the several dimensions, the Methods also include:

obtaining a sample medical image, wherein the sample medical image includes the actual area of the lesion;

Utilize described feature extraction sub-network to carry out feature extraction to described sample medical image, obtain the first sample feature map of several dimensions;

Taking the first sample feature map of a preset dimension as a reference sample feature map, the reference sample feature map is used to generate a lesion sample probability map, wherein the lesion sample probability map is used to represent the sample medical image. The probability that different regions belong to lesions;

Using the fusion processing sub-network to fuse the lesion sample probability map with the first sample feature maps of the several dimensions to obtain the final fused sample feature map;

Use the fusion processing sub-network to perform detection processing on the final fusion sample feature map to obtain the detection area about the lesion in the sample medical image;

Using the difference between the actual area and the detection area, the network parameters of the image detection model are adjusted.
The method according to claim 11, wherein the adjusting the network parameters of the image detection model using the difference between the actual area and the detection area comprises:

The actual area and the detection area are processed by using an aggregate similarity loss function to determine the loss value of the image detection model;

The network parameters of the image detection model are adjusted with a predetermined learning rate using the loss value.
An image detection device, comprising:

The image acquisition module is used to acquire the medical image to be tested;

a feature extraction module, configured to perform feature extraction on the medical image to be tested to obtain a first feature map of several dimensions;

The image generation module is configured to use the first feature map of a preset dimension as a reference feature map, and use the reference feature map to generate a lesion probability map, wherein the lesion probability map is used to represent the medical image to be tested. The probability that the different regions of , belong to the lesion;

an image fusion module, configured to fuse the lesion probability map with the first feature maps of the several dimensions to obtain a final fusion feature map;

A detection processing module is used to perform detection processing on the final fusion feature map to obtain a detection result about the lesion in the medical image to be tested.
An electronic device comprises a mutually coupled memory and a processor, the processor is configured to execute program instructions stored in the memory, so as to implement the image detection method according to any one of claims 1 to 12.
A computer-readable storage medium having program instructions stored thereon, when the program instructions are executed by a processor, the image detection method according to any one of claims 1 to 12 is implemented.
A computer program, comprising computer-readable codes, when the computer-readable codes are executed in an electronic device, a processor in the electronic device executes the image detection method for implementing any one of claims 1 to 12 .