WO2023198166A1 - 图像检测方法、系统、装置及存储介质 - Google Patents

图像检测方法、系统、装置及存储介质 Download PDF

Info

Publication number
WO2023198166A1
WO2023198166A1 PCT/CN2023/088226 CN2023088226W WO2023198166A1 WO 2023198166 A1 WO2023198166 A1 WO 2023198166A1 CN 2023088226 W CN2023088226 W CN 2023088226W WO 2023198166 A1 WO2023198166 A1 WO 2023198166A1
Authority
WO
WIPO (PCT)
Prior art keywords
attention map
model
lesion
probability
attention
Prior art date
Application number
PCT/CN2023/088226
Other languages
English (en)
French (fr)
Inventor
吴青霞
刘晓鸣
Original Assignee
北京联影智能影像技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京联影智能影像技术研究院 filed Critical 北京联影智能影像技术研究院
Publication of WO2023198166A1 publication Critical patent/WO2023198166A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • This specification relates to the field of medical image processing, and in particular to an image detection method, system, device and storage medium.
  • Medical imaging is often used for screening and diagnosis of diseases, especially early detection and diagnosis of various cancers.
  • Manual or computer-assisted detection is usually used to determine the lesions and the extent of cancer development (for example, benign and malignant, early and late stages, etc.).
  • breast cancer Take breast cancer as an example.
  • Mammography screening is a common method for breast cancer screening.
  • doctors need to manually judge the breast cancer based on the imaging signs of the mammography image (for example, the nature and shape of masses and calcifications, whether the nipple is indented, etc.) benign and malignant tumors.
  • the judgment based on imaging signs lacks quantitative standards, the judgment standards of different doctors are inconsistent, and the diagnosis based on imaging signs has problems such as low sensitivity and specificity of lesion detection.
  • One embodiment of this specification provides an image detection method.
  • the method includes: acquiring a plurality of segmented image blocks in an initial medical image of the target object; and generating at least one attention map of the initial medical image based on the plurality of segmented image blocks, where the at least one attention map includes At least one of a first attention map and a second attention map, the first attention map is related to the location of the lesion, and the second attention map is related to the classification of the lesion; based on the at least one attention map, the target object is generated test results.
  • the first attention map includes a first attention value of an element in the initial medical image, the first attention value is related to a probability that the element belongs to a lesion, and the second attention value The attempt includes a second attention value for an element in the initial medical image, the second attention value being related to a lesion classification of the element.
  • a first model may be used to generate the lesion location probability and the lesion classification probability of the segmented image block, wherein the first model is The trained machine learning model; obtains the first attention map and the second attention map based on the lesion location probability and the lesion classification probability.
  • the first attention map may be generated based on the position information of each segmented image block and the lesion location probability; based on the position information of each segmented image block and the lesion classification Probabilistically generate the second attention map.
  • the lesion location probability and the lesion classification probability can be remapped to the initial medical image to obtain the first attention map and the second attention map.
  • the mapping may include at least one of a linear mapping and a Gaussian mapping.
  • a weight corresponding to each segmented image block in the plurality of segmented image blocks may be obtained, wherein the first attention map and the second attention map may be further generated based on the weight.
  • the plurality of divided image blocks may include at least one first divided image block of a first size and at least one second divided image block of a second size, and the at least one first divided image block may For generating the lesion location probability, the at least one second segmented image block may be used for generating the lesion classification probability.
  • the first model may include an encoder, a decoder, and a classification framework.
  • the encoder can be used to extract the coding feature information of the divided image block;
  • the decoder can be used to extract the decoding of the divided image block based on the coding feature information.
  • feature information and generate the lesion location probability of the segmented image block based on the decoded feature information;
  • the classification framework may be used to generate the segmentation based on at least a part of the encoded feature information and the decoded feature information. The lesion classification probability of the image patch.
  • the first model may further include a normalization layer, wherein the normalization layer may be used to connect the encoder and the decoder and normalize the encoding feature information. Unified processing.
  • the decoder may include an attention block for enhancing the spatial features and the channel features based on the weights of the spatial features, the weights of the channel features and the original features. .
  • a second model may be used to generate the detection result of the target object based on the at least one attention map and the initial medical image, where the second model is a trained machine learning model.
  • the initial medical image may include multiple initial medical images corresponding to multiple viewing angles
  • the at least one attention map may include at least one attention map corresponding to each of the initial medical images.
  • the second model can be used to process the initial medical image and its corresponding at least one attention map to obtain the initial medical image.
  • Initial detection results based on the initial detection results of the multiple initial medical images, generate a detection result of the target object.
  • the at least one attention map can be obtained through a first model
  • the detection result of the target object can be obtained through a second model
  • the first model and the second model can be trained in the following manner Obtain: obtain segmented image block samples of medical image samples; use the segmented image block samples as training samples for the first model, train an initial first model, and obtain the first model; based on the medical image samples and the Use the first model to generate attention map samples corresponding to the medical image samples; use the attention map samples and the medical image samples as training samples for the second model, train the initial second model, and obtain the second Model.
  • an image detection system including an image block acquisition module, an attention map generation module and a detection result generation module;
  • the image block acquisition module is used to acquire multiple segmented images in an initial medical image of a target object. block;
  • the attention map generation module is configured to generate at least one attention map of the initial medical image based on the plurality of segmented image blocks, the at least one attention map including a first attention map and a second attention map.
  • the first attention map is related to the location of the lesion
  • the second attention map is related to the classification of the lesion;
  • the detection result generation module is used to generate the detection of the target object based on the at least one attention map. result.
  • One embodiment of this specification provides an image detection system, including at least one memory that stores at least one set of computer instructions; and at least one processor that communicates with the memory.
  • the processor executes the following method: obtains a plurality of segmented image blocks in the initial medical image of the target object; and generates at least one attention map of the initial medical image based on the plurality of segmented image blocks.
  • the at least one attention map includes at least one of a first attention map and a second attention map, the first attention map is related to lesion location, and the second attention map is related to lesion classification; based on the at least one Draw an attention map to generate detection results of the target object.
  • One embodiment of this specification provides an image detection device, including a processor, where the processor is configured to execute the image detection method.
  • One embodiment of this specification provides a computer-readable storage medium.
  • the storage medium stores computer instructions. After the computer reads the computer instructions in the storage medium, the computer executes the image detection method.
  • Figure 1 is a schematic diagram of an application scenario of an image detection system according to some embodiments of this specification.
  • Figure 2 is a schematic diagram of an image detection system according to some embodiments of this specification.
  • Figure 3 is an exemplary flow chart of an image detection method according to some embodiments of this specification.
  • Figure 4 is an exemplary flow chart of an image detection method according to some embodiments of this specification.
  • Figure 5 is an exemplary flow chart of an image detection method according to some embodiments of this specification.
  • Figure 6 is a schematic diagram of an image detection method according to some embodiments of this specification.
  • Figure 7 is an exemplary flow chart of a model training method according to some embodiments of this specification.
  • Figure 8 is a schematic structural diagram of a first model according to some embodiments of this specification.
  • Figure 9 is a schematic diagram of an initial medical image according to some embodiments of the present specification.
  • system means of distinguishing between different components, elements, parts, portions or assemblies at different levels.
  • said words may be replaced by other expressions if they serve the same purpose.
  • Figure 1 is a schematic diagram of an application scenario of an image detection system according to some embodiments of this specification.
  • the image detection system 100 may include a medical imaging device 110 , a first computing device 120 , a second computing device 130 , a user terminal 140 , a storage device 150 and Network 160.
  • Medical imaging equipment 110 may refer to a device that uses different media to reproduce the internal structure of a target object (such as the human body) into images.
  • the medical imaging device 110 may be any device that can image or treat a designated body part of a target object (such as a human body), such as a mammography machine, MRI (Magnetic Resonance Imaging), CT ( Computed Tomography), PET (Positron Emission Tomography), ultrasonic diagnostic equipment, etc.
  • MRI Magnetic Resonance Imaging
  • CT Computed Tomography
  • PET Positron Emission Tomography
  • ultrasonic diagnostic equipment etc.
  • the medical imaging device 110 provided above is for illustrative purposes only and is not a limitation on its scope.
  • the medical imaging device 110 can acquire medical images (eg, magnetic resonance (MRI) images, CT images, ultrasound images, mammography images) of designated parts of the patient (eg, breast, brain, stomach, etc.) etc.) and sent to other components of the system 100 (eg, the first computing device 120, the second computing device 130, the storage device 150). In some embodiments, medical imaging device 110 may exchange data and/or information with other components in system 100 over network 160 .
  • medical images eg, magnetic resonance (MRI) images, CT images, ultrasound images, mammography images
  • medical imaging device 110 may exchange data and/or information with other components in system 100 over network 160 .
  • the first computing device 120 and the second computing device 130 are systems with computing and processing capabilities, which may include various computers, such as servers and personal computers, or may be a computing platform composed of multiple computers connected in various structures. In some embodiments, the first computing device 120 and the second computing device 130 may be the same device or different devices.
  • One or more sub-processing devices may be included in the first computing device 120 and the second computing device 130, and the processing devices may execute the program instructions.
  • processing devices may include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits, ASIC), or other types of integrated circuits.
  • the first computing device 120 can process information and data related to medical images.
  • the first computing device 120 can execute an image detection method as shown in some embodiments of this specification to obtain at least one image detection result, that is, a detection result of a target object (eg, a patient, etc.).
  • Exemplary test results may include at least one of benign and malignant breast diagnosis results, breast BI-RADS diagnosis results, breast density diagnosis results, breast calcification diagnosis results, breast mass property diagnosis results, etc.
  • the first computing device 120 can obtain the disease detection results (eg, a lesion location probability map and a lesion classification probability map) through a machine learning model.
  • the first computing device 120 may obtain the trained machine learning model from the second computing device 130 .
  • the first computing device 120 may determine the detection result of the target object, that is, the diagnosis result of the disease based on the lesion location probability map and the lesion classification probability map. In some embodiments, first computing device 120 may exchange information and data over network 160 and/or other components in system 100 (eg, medical imaging device 110, second computing device 130, user terminal 140, storage device 150) . In some embodiments, first computing device 120 may connect directly with second computing device 130 and exchange information and/or data.
  • the second computing device 130 can be used for model training.
  • the second computing device 130 can execute the training method of a machine learning model as shown in some embodiments of this specification to obtain a trained machine learning model.
  • the second computing device 130 may obtain image information from the medical imaging device 110 as training data for the model.
  • first computing device 120 and the second computing device 130 may also be the same computing device.
  • the user terminal 140 may receive and/or display the processing results of the medical image.
  • the user terminal 140 may receive an image detection result of the medical image from the first computing device 120, and diagnose and treat the patient based on the image detection result.
  • the user terminal 140 can cause the first computing device 120 to perform the image detection method as shown in some embodiments of this specification through instructions.
  • the user terminal 140 can control the medical imaging device 110 to obtain medical images of a specific part.
  • the user terminal 140 may be one of a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, a desktop computer, or other devices with input and/or output functions, or the like. random combination.
  • Storage device 150 may store data or information generated by other devices.
  • the storage device 150 may store medical images collected by the medical imaging device 110 .
  • the storage device 150 may store data and/or information processed by the first computing device 120 and/or the second computing device 130, such as a lesion location probability map, a lesion classification probability map, an attention map, etc.
  • the storage device 150 may include one or more storage components, and each storage component may be an independent device or a part of other devices. Storage devices can be local or via the cloud.
  • Network 160 may connect components of the system and/or connect parts of the system to external resources.
  • Network 160 enables communication between components and with other components outside the system, and facilitates the exchange of data and/or information.
  • one or more components in system 100 eg, medical imaging device 110, first computing device 120, second computing device 130, user terminal 140, storage device 150
  • network 160 may be any one or more of a wired network or a wireless network.
  • first computing device 120 and/or the second computing device 130 may be based on a cloud computing platform, such as public cloud, private cloud, community and hybrid cloud, etc.
  • a cloud computing platform such as public cloud, private cloud, community and hybrid cloud, etc.
  • Figure 2 is a schematic diagram of an image detection system according to some embodiments of this specification.
  • the image detection system 200 may include an image block acquisition module 210 , an attention map generation module 220 , and a detection result generation module 230 .
  • the image patch acquisition module 210 may be used to acquire multiple segmented image patches in the initial medical image of the target object.
  • the attention map generation module 220 may be used to generate at least one attention map of the initial medical image based on a plurality of segmented image blocks.
  • at least one attention map may include at least one of a first attention map and a second attention map, the first attention map is related to lesion localization, and the second attention map is related to lesion classification.
  • the first attention map may include a first attention value of an element in the initial medical image
  • the first attention value may be related to a probability that the element belongs to the lesion
  • the second attention map may include a first attention value of the element in the initial medical image.
  • the second attention value of the element can be related to the focus classification of the element.
  • the attention map generation module 220 may use a first model to generate a lesion localization probability and a lesion classification probability of the segmented image block, where the first model may be Trained machine learning model.
  • the first model may include an encoder, a decoder, and a classification framework.
  • the encoder can be used to extract the coding feature information of the segmented image block
  • the decoder can be used to extract the decoding feature information of the segmented image block based on the coding feature information, and generate the segmented image block based on the decoding feature information.
  • the lesion localization probability; the classification framework may be used to generate a lesion classification probability of the segmented image block based on at least a part of the encoded feature information and the decoded feature information.
  • the first model may also include a normalization layer.
  • the normalization layer can be used to connect the encoder and decoder and normalize the encoding feature information.
  • the decoder may include attention blocks.
  • the attention block can be used to enhance spatial features and channel features based on the weight of spatial features, the weight of channel features and original features.
  • the attention map generation module 220 may obtain the first attention map and the second attention map based on the lesion location probability and the lesion classification probability.
  • the attention map generation module 220 may generate a first attention map based on the position information and lesion location probability of each segmented image block; and generate a second attention map based on the position information and lesion classification probability of each segmented image block.
  • the attention map generation module 220 may obtain the weight corresponding to the segmented image block, wherein the first attention map and the second attention map may be further based on the weight. generate.
  • the plurality of segmented image blocks may include at least one first segmented image block of a first size and at least one second segmented image block of a second size, and the at least one first segmented image block may be used to generate the lesion location. probability, at least one second segmented image patch may be used to generate a lesion classification probability.
  • the attention map generation module 220 may remap the lesion location probability and the lesion classification probability to the initial medical image to obtain the first attention map and the second attention map, where the mapping may include linear mapping and Gaussian mapping. At least one.
  • the detection result generation module 230 may be used to generate a detection result of the target object based on at least one attention map.
  • the detection result generation module 230 may use a second model to generate a detection result of the target object based on at least one attention map and the initial medical image, where the second model may be a trained machine learning model.
  • the initial medical image may include multiple initial medical images corresponding to multiple viewing angles
  • the at least one attention map may include at least one attention map corresponding to each initial medical image.
  • the detection result generation module 230 can use the second model to process the initial medical image and its corresponding at least one attention map to obtain an initial detection result of the initial medical image.
  • the detection result generation module 230 may generate detection results of the target object based on the initial detection results of these initial medical images.
  • the image detection system 200 may also include a model training module (not shown in the figure).
  • the model training module can obtain segmented image block samples of medical image samples; use the segmented image block samples as training samples for the first model, train the initial first model, and obtain the first model; based on the medical image samples and the first model, generate Attention map samples corresponding to medical image samples; use the attention map samples and medical image samples as training samples for the second model, train the initial second model, and obtain the second model.
  • image detection system 200 may be implemented by first computing device 120 and/or second computing device 130 .
  • the image patch acquisition module 210, the attention map generation module 220, and the detection result generation module 230 may be implemented by the first computing device 120.
  • the model training module may be implemented by the second computing device 130.
  • Figure 3 is an exemplary flow chart of an image detection method according to some embodiments of this specification.
  • process 300 includes the following steps. In some embodiments, process 300 may be performed by first computing device 120.
  • Step 310 Obtain multiple segmented image blocks in the initial medical image of the target object.
  • step 310 may be performed by image patch acquisition module 210.
  • the target objects are those who need to obtain examination results, such as diagnosed patients, examination subjects undergoing cancer screening, etc.
  • the target object may include specific parts of the human body, such as the breast, brain, stomach, etc.
  • the target object may include a simulated object, such as a digital human body simulated by a computer, etc.
  • the initial medical image is an original medical image used to obtain the detection result of the target object, for example, MRI image, CT image, ultrasound image, mammography image, etc.
  • the initial medical image may be obtained by scanning the target object with a medical imaging device (eg, medical imaging device 110).
  • the initial medical image can be obtained through other methods, for example, from a storage device, through a simulated scan of a digital human body, etc.
  • the first computing device 120 may perform preliminary screening on the initial medical images to remove images that do not meet quality requirements (eg, insufficient clarity, do not contain a region of interest, etc.).
  • quality requirements eg, insufficient clarity, do not contain a region of interest, etc.
  • the image may be considered to be too slender, and the first computing device 120 may use padding on the initial medical image.
  • a preset value eg, 10:1, etc.
  • Segmented image blocks refer to the blocks of images obtained by segmenting/blocking medical images.
  • the initial medical images collected by medical imaging equipment are generally large-size images with a large amount of information. If medical analysis is performed directly through the initial medical images, it will not only increase the pressure on the computing equipment to process the data, but also cause excessive information content in the images. It is difficult to carry out targeted analysis, resulting in inaccurate analysis results.
  • the first computing device 120 may perform the initial calculation in various ways (eg, sliding window method, random sampling method, etc.) Medical images are divided into blocks to obtain multiple segmented image blocks.
  • the first computing device 120 may obtain the plurality of segmented image blocks of the initial medical image from other devices (eg, other computing devices, storage devices, etc.).
  • the initial medical image may also include useless information (e.g., background, other body parts, etc.) that is irrelevant to the region of interest, which may cause the computing device to There is a lot of ineffective work when processing data, which affects the analysis results.
  • the area of interest may be difficult to distinguish from other parts of the image, thus affecting subsequent processing.
  • the first computing device 120 may divide the region of interest based on image information (eg, pixel information/grayscale information, RGB information, depth information, etc.) in the initial medical image. The corresponding parts are segmented to obtain the initial medical image corresponding to the area of interest.
  • Figure 9 is a schematic diagram of an initial medical image according to some embodiments of this specification, in which the region of interest is the breast.
  • the left image in Figure 9 is an unprocessed initial medical image. It can be seen that it includes a large number of black background areas, which is useless information. If the background area also participates in blocking, it will lead to a lot of invalid work, and the boundaries between the breast outline and other surrounding areas will not be obvious and difficult to distinguish.
  • the image on the right side of Figure 9 is the initial medical image obtained by segmenting the breast. It can be seen that a large amount of useless information has been segmented, and the breast outline is obvious and the boundaries are clear.
  • the initial medical image obtained by scanning and the medical image obtained by segmenting the region of interest on the initial medical image are collectively referred to as the initial medical image in the following text.
  • the first computing device 120 may segment the initial medical image based on the image information. For example, the pixels based on the initial medical image are divided according to a preset image block size, that is, it is divided into image blocks of a preset size. For another example, the RGB information based on the initial medical image is divided according to the preset RGB range and divided into image blocks of different RGB ranges. For another example, the depth information based on the initial medical image is divided according to a preset depth range and divided into image blocks of different depth ranges. In some embodiments, the first computing device 120 may tile the initial medical image through other means. For example, image calibration is performed in the initial medical image through detection frames to obtain multiple image detection frames, etc.
  • the first computing device 120 may divide the initial medical image into image blocks of various sizes, eg, fixed size image blocks, a combination of multiple size image blocks, etc.
  • the image blocks may include two-dimensional image blocks (eg, with dimensions of 512*512), three-dimensional image blocks (eg, with dimensions of 512*512*512), and the like.
  • the first computing device 120 may select a preset number (eg, 125, 225, etc.) of segmented image blocks from all segmented image blocks of the initial medical image for further processing.
  • the size of the initial medical image is 512*512
  • it can be divided into blocks in one of the following ways: split into image blocks of size 64*64, and randomly select 125 segmented images of size 64*64 Blocks; split into image blocks of size 64*64 and image blocks of size 32*32, randomly select 125 image blocks of size 64*64 and 125 image blocks of size 32*32.
  • the blocking strategy can be determined in various ways. For example, different blocking strategies are set according to the analysis requirements and the actual situation of the initial medical image, so as to perform different processing on different initial medical images. This is to improve the analysis efficiency of computing equipment while ensuring the quality of analysis.
  • the first computing device 120 may set a larger image block size, so that the number of segmented image blocks is larger. Less, it is easier for the computing device to quickly process the segmented image blocks.
  • the first computing device 120 may select a corresponding model (eg, a first model, a second model) based on the size of the segmented image block to perform subsequent processing based on the segmented image block. For example, if the image block size is 64*64, select a model whose input image size is 64*64 for processing.
  • a corresponding model eg, a first model, a second model
  • the first computing device 120 may segment the initial medical image multiple times, and each segmentation generates image blocks of a specific size for subsequent analysis, where the objects of each segmentation are unsegmented objects.
  • initial medical image For example, image blocks of different sizes can be used for a lesion positioning operation (abbreviated as lesion positioning) and a lesion classification operation (abbreviated as lesion classification) respectively.
  • lesion positioning abbreviated as lesion positioning
  • lesion classification abbreviated as lesion classification
  • multiple operations can be performed using image blocks of different sizes.
  • the first computing device 120 may generate a plurality of image tiles of a first size (eg, 512*512, etc.) and a plurality of image tiles of a second size (eg, 256*256, etc.), the first size
  • the image block of the second size is used for subsequent lesion localization (for example, generating a lesion localization probability), and the image block of the second size is used for subsequent lesion classification (for example, generating a lesion classification probability).
  • lesion localization for example, generating a lesion localization probability
  • lesion classification probability for example, generating a lesion classification probability
  • the first computing device 120 can use segmented image blocks of different sizes to perform the same operation (for example, any one of lesion localization, lesion classification, etc.) multiple times, and then integrate the multiple results obtained. Get the result of this operation. For example, for lesion localization, you can use an image block of the first size (for example, 512*512, etc.) to perform an operation, and then use an image block of the second size (for example, 256*256, etc.) to perform an operation, and then perform the operation twice The lesion localization results are combined. Because image patches of different scales can provide different scale information, this operation can improve the accuracy of the lesion location results and lesion classification results.
  • the first computing device 120 when combining the lesion location results corresponding to image blocks of different sizes, can set a corresponding weight, which is related to the accuracy of the identification result provided by the size of the image block. For example, an image block with a size of 512*512 has a better effect on localizing lesions and has a larger weight, while an image block with a size of 256*256 has a smaller weight.
  • adjacent segmented image blocks may overlap.
  • the overlapping area between adjacent image blocks accounts for 25% of the total area of each image block.
  • Step 320 Generate at least one attention map of the initial medical image based on multiple segmented image blocks.
  • at least one attention map includes at least one of a first attention map and a second attention map, the first attention map is related to lesion localization, and the second attention map is related to lesion classification.
  • step 320 may be performed by attention map generation module 220.
  • the first attention map and the second attention map are data used to identify the disease analysis results of the target object, for example, images that identify the probability of belonging to a lesion, images that identify the probability of a lesion type, etc.
  • the first attention map may include first attention values for elements (eg, pixels, voxels, etc.) in the initial medical image, where the first attention values may be used for lesion localization, which are related to the elements
  • the probability of belonging to a lesion is related.
  • the first attention value may be related to the probability of breast cancer, the probability of breast mass, the probability of breast calcification, the probability of breast blood vessel calcification, etc.
  • the second attention map may include a second attention value for an element in the initial medical image, wherein the second attention value may be used for lesion classification, which is related to the lesion classification of the element.
  • the second attention value may be related to the benign and malignant classification probability of the breast, the classification probability of the calcification properties of the breast, the classification probability of the breast mass properties, the classification probability of the breast lymph node properties, the classification probability of the breast soft tissue structure properties, etc.
  • the first attention map and the second attention map may be represented by multi-dimensional values (eg, pseudo-color images, etc.).
  • the first computing device 120 may process the segmented image blocks through the first model to obtain the first attention map and/or the second attention map.
  • the first computing device 120 may process the segmented image blocks through the first model to obtain the first attention map and/or the second attention map.
  • the first computing device 120 may obtain the first attention map and/or the second attention map through other methods, for example, through manual annotation.
  • Step 330 Generate a detection result of the target object based on at least one attention map.
  • step 330 may be performed by the detection result generation module 230.
  • the detection result of the target object refers to the diagnosis result of the disease of the target object obtained based on the initial medical image, for example, the benign/malignant discrimination result of cancer, the nature of the mass, the type of fracture, etc.
  • the detection results can include the diagnosis results of benign and malignant breasts, the diagnosis results of breast density, the diagnosis results of BI-RADS classification of breasts, the diagnosis results of breast calcification, and the diagnosis results of the nature of breast masses. Any of these.
  • the detection results may include a determination of the image type for the initial medical image.
  • the detection result is that the image type of the initial medical image is the first type, and the first type indicates that the initial medical image is a breast nipple indentation type image.
  • the image type of the initial medical image is the second type, and the second type indicates that the initial medical image is an image with high breast gland density.
  • the first computing device 120 may process the attention maps through the second model to obtain the detection result of the target object.
  • the relevant description in Figure 5 please refer to the relevant description in Figure 5, which will not be described again here.
  • the first computing device 120 may determine the detection result of the target object based on the lesion location probability and/or the lesion classification probability corresponding to the element in the attention map. For example, if the detection result is whether the target object has a tumor, and the first attention map contains the probability that each element belongs to a tumor lesion (lesion localization probability), the first computing device 120 may calculate the probability exceeding a threshold (for example, 60%) the number of elements. If the number exceeds the threshold (for example, 10,000), it is determined that the target object has a tumor.
  • a threshold for example, 60%
  • the first computing device 120 can count the number of elements whose malignancy probability exceeds a threshold (eg, 80%). If the number exceeds a threshold (for example, 5000), the tumor of the target object is determined to be malignant.
  • a threshold for example, 5000
  • the first computing device 120 may determine whether the target object has breast cancer based on the breast cancer probability in the first attention map and the breast density probability in the second attention map. cancer.
  • the first computing device 120 may obtain the detection result of the target object through other methods, for example, by manually processing the attention map.
  • Figure 4 is an exemplary flowchart of an image detection method according to some embodiments of this specification.
  • process 400 includes the following steps.
  • process 400 may be performed by first computing device 120 or attention map generation module 220.
  • the first computing device 120 may generate at least one attention map of the initial medical image based on a plurality of segmented image blocks.
  • Step 410 For each segmented image block among the plurality of segmented image blocks, use the first model to generate the lesion location probability and the lesion classification probability of the segmented image block.
  • the first computing device 120 may input the segmented image block into the first model to obtain the output lesion location probability and lesion classification of the segmented image block. Probability, where the first model is a trained machine learning model.
  • the output of the first model may include a lesion localization probability and a lesion classification probability for each element (eg, pixel, voxel, etc.) in the segmented image block.
  • the lesion location probability represents the probability that an object (eg, image block, element in the image patch, etc.) belongs to a certain lesion (eg, adenocarcinoma, breast mass, breast calcification, breast vascular calcification, etc.).
  • the lesion localization probability of the segmented image block may be composed of the lesion localization probability of the elements in the image block.
  • the lesion location probability may include various expression forms, such as numbers, tables, databases, probability images, etc.
  • the lesion location probability can be expressed as a percentage, a level, etc., for example, 20%, level I (probability less than 10%), level C (probability greater than or equal to 60% and less than 80%), etc.
  • the first model analyzes multiple segmented image blocks, it outputs that the lesion location probability of element A in the first segmented image block is 10%, and the lesion location probability of element B in the second segmented image block is 60%, then It can be expressed that the probability that element A in the first segmented image block belongs to the lesion is 10%, and the probability that element B in the second segmented image block belongs to the lesion is 60%.
  • the lesion location probability may include various types.
  • the lesion location probability may be represented by a lesion location probability map.
  • different types of lesion location probabilities can correspond to different lesion location probability maps.
  • Lesion classification probability represents the probability that an object (e.g., image block, element in image block, etc.) belongs to a specific type of lesion, for example, the probability of benign or malignant breast mass, whether the breast mass is tumor/non-tumor Probability etc.
  • the lesion classification probability of the segmented image block may be composed of the lesion classification probability of the elements in the image block.
  • the lesion classification probability may include various expression forms, such as numbers, tables, databases, probability images, etc.
  • the lesion classification probability may be expressed as a percentage, a grade, etc.
  • the first model outputs that the probability that element A in the first segmented image block is a tumor is 90% and the probability of being benign is 70%; the probability that element B in the second segmented image block is a tumor is output The probability is 30%, and the probability of benignity is 90%.
  • the lesion classification probabilities may include various types. Taking breast disease as an example, it may include at least one of the classification probabilities of benign and malignant breasts, the classification probabilities of breast calcification properties, the classification probabilities of breast mass properties, the classification probabilities of breast lymph node properties, the classification probabilities of breast soft tissue structural properties, etc. .
  • the lesion classification probability may be represented by a lesion classification probability map, wherein different types of lesion classification probabilities may correspond to different lesion classification probability maps.
  • the lesion type may include various types. Taking breast disease as an example, it may include at least one of whether it is a lesion, breast concave and convex type, breast gland density type, breast BI-RADS level type, etc. Unless otherwise specified in the following parts of this manual, the lesion location probability will be represented by the lesion location probability map, and the lesion classification probability will be represented by the lesion classification probability map. This is for illustration only and not as a limitation.
  • the lesion location probability map and the lesion classification probability map may be represented using multidimensional values (eg, pseudo-color images, etc.).
  • the first model may obtain the lesion location probability by comparing the segmented image blocks with image blocks with lesions and/or image blocks without lesions. In some embodiments, the first model may obtain the lesion classification probability and/or the lesion type by comparing the segmented image blocks with image blocks having a specific type of disease.
  • the plurality of divided image blocks may include at least one first divided image block of a first size and at least one second divided image block of a second size.
  • at least one first segmented block can be used to generate a lesion location probability
  • at least one second segmented image block can be used to generate a lesion classification probability.
  • the first size and the second size may be the same, that is, the size of the first segmented image block and the second segmented image block are the same.
  • the size of the first segmented image block used to generate the lesion location probability and the second segmented image block used to generate the lesion classification probability are both 512*512.
  • the first size and the second size may be different, that is, the size of the first segmented image block and the second segmented image block are different.
  • the first size of the first segmented image block used to generate the lesion location probability may be 512*512
  • the second size of the second segmented image block used to generate the lesion classification probability may be 256*256.
  • the first model may include at least two different branches or single-task models to process image patches of different sizes respectively. Among them, one branch/single-task model can be used to process image blocks of a first size, and another branch/single-task model can be used to process image blocks of a second size.
  • the first model may include multiple layers (layers), wherein the size of image blocks processed by each layer may be different, and these layers may be connected through operations such as downsampling or upsampling.
  • the plurality of divided image blocks may include at least one first divided image block of a first size and at least one second divided image block of a second size.
  • at least one of the lesion localization probability and the lesion classification probability may be generated based on at least one first segmented image block and at least one second segmented image block simultaneously.
  • either one of the lesion localization probability and the lesion classification probability is generated based on the first segmented image block of 512*512 and the second segmented image block of 256*256.
  • the first model may be a convolutional neural network (Convolutional Neural Networks, CNN) such as U-Net.
  • the first model may include various machine learning models, such as convolutional neural networks, fully convolutional networks (Fully Convolutional Networks, FCN) and other neural network models.
  • the first model may be a multi-task model, and may output multiple processing results at the same time, for example, simultaneously output the lesion location probability and the lesion classification probability.
  • the first model may be composed of two single-task models, wherein one model is used to output a lesion localization result (eg, lesion location probability), and the other model is used to output a lesion classification result (eg, lesion classification probability), the inputs to both models are the same.
  • the structure of the first model may be as shown in the first model 800 in Figure 8. For details, please refer to the relevant description of Figure 8, which will not be described again here.
  • the first model can be obtained by unsupervised/supervised training of an initial first model (ie, an untrained first model) including image patches with lesions and image patches without lesions.
  • the first model can be obtained by training the initial first model through a single algorithm, an integrated algorithm, image patches with lesions, and image patches without lesions.
  • an initial first model ie, an untrained first model
  • the first model can be obtained by training the initial first model through a single algorithm, an integrated algorithm, image patches with lesions, and image patches without lesions.
  • the first computing device 120 may obtain the first attention map and the second attention map based on the focus location probability and the lesion classification probability.
  • the first computing device 120 may remap the lesion location probability and the lesion classification probability to the initial medical image, thereby obtaining the first attention map and the second attention map.
  • the mapping method can adopt multiple mapping methods such as linear mapping and Gaussian mapping.
  • the first computing device 120 may generate the first attention map and the second attention map based on the first attention map, the second attention map, and the position information of the segmented image blocks by performing steps 420 and 430 .
  • Step 420 Generate a first attention map based on the position information and lesion location probability of each segmented image block.
  • the first computing device 120 may combine all the lesion location probabilities according to the position information of each segmented image block, thereby obtaining the first attention map.
  • the first computing device 120 may obtain the weight corresponding to each segmented image block, and generate the first attention map based on the weight of each segmented image block and the lesion location probability.
  • the weight can be represented by various forms such as numbers, tables, databases, and weight diagrams (for example, Gaussian probability diagrams).
  • the weight map may include a weight value of an element in the image block, and the weight value may reflect the confidence of the model output result (ie, lesion location probability) of the element.
  • the model's output results for the center area of the image are more accurate, while the output results for the edge areas are less accurate.
  • the weight value of the middle area is usually high and the weight value of the edge area is low.
  • the first computing device 120 may cause the weights to be the middle region The value is larger and the value in the edge area is smaller.
  • weights corresponding to different segmented image blocks may be the same.
  • the first computing device 120 may update the lesion location probability of the segmented image block based on its corresponding weight to obtain an updated lesion location probability. Specifically, the first computing device 120 can multiply the weight of the image block and the lesion location probability, that is, multiply the weight of the element in the image block by its corresponding lesion location probability to obtain the updated focus location probability of the element.
  • the updated lesion location probability of the image patch may include updated lesion location probabilities of the elements in the image patch.
  • the first computing device 120 may combine the updated lesion location probabilities of each segmented image block according to the position information of each segmented image block in the initial medical image to obtain the first attention corresponding to the initial medical image. Try.
  • the first computing device 120 can determine the corresponding image block where the element is located based on the position information of the image block, and the update of the element in the corresponding image block. The corresponding value in the subsequent lesion location probability. After determining the corresponding value of each element, all corresponding values can be summed and averaged to obtain the first attention value of the element. After the first attention value of each element is determined, the first attention map can be generated.
  • the weights and positions of the image patches in the original medical image may be stored in a lookup table.
  • the lesion location probability, the updated focus location probability, the first attention map, etc. can be presented in the form of pseudo-color images, and the element values therein can be reflected by RGB values.
  • lesion localization probabilities are presented with raw pseudocolor images.
  • the first computing device 120 may convert the lesion location probability of each image block into an original pseudo-color image according to the corresponding relationship between the lesion location probability and the RGB value.
  • the first computing device 120 may multiply the original pseudo-color image of the image block and the weight to obtain an updated pseudo-color image (ie, the updated lesion location probability).
  • the first computing device 120 can find the RGB value corresponding to the element in the updated pseudo-color image, sum and average the RGB values of the element, and obtain the final value of the element. RGB value.
  • the final RGB value of each element can form a pseudo-color image corresponding to the first attention map, where the RGB value of each element can correspond one-to-one with its first attention value.
  • the correspondence between the lesion location probability and the RGB value can be stored using an information correspondence table.
  • the first computing device 120 may directly generate the first attention map based on the location information and the lesion location probability of each image block. Specifically, the first computing device 120 can directly combine the lesion location probabilities of each segmented image block according to the position information of each segmented image block in the initial medical image to obtain the first attention map corresponding to the initial medical image.
  • the first computing device 120 can determine the image block in which the element is located, the lesion location probability of the element in the image block, and the location probability of the element based on the position of the image block. All probabilities are summed and averaged to obtain the first attention value corresponding to the element.
  • Step 430 Generate a second attention map based on the position information and lesion classification probability of each segmented image block.
  • the first computing device 120 may determine each segmented image block in the initial medical image based on the position information, the corresponding weight map and the lesion classification probability. The second attention value of the element is obtained, thereby obtaining the second attention map.
  • the first computing device 120 may directly generate the second attention map based on the location information and lesion classification probability of each segmented image block.
  • the specific method of generating the second attention map based on the position information of each segmented image block and the lesion classification probability is similar to the method of generating the first attention map. The difference is that the first attention map is generated based on the lesion location probability.
  • the first attention map is generated based on the lesion location probability.
  • the first computing device 120 may input each segmented image block and its position information into the first model to obtain an output first attention map and a second attention map.
  • the step of generating the first attention map and the second attention map based on the position information, lesion location probability and lesion classification probability of each segmented image block is integrated in the first model.
  • the first computing device 120 may input the initial medical image into the first model to obtain an output first attention map and a second attention map. That is, based on the previous embodiment, the step of dividing the image into blocks is further integrated into the first model.
  • the first computing device 120 may synthesize the probabilities to generate a first attention map and/or a second attention map.
  • the first computing device 120 may select specific lesion localization probabilities and lesion classification probabilities according to the final goal to be achieved for generating the first attention map and/or the second attention map. For example, if the ultimate goal is to identify breast cancer, and the correlation between breast cancer and breast gland density is high, you can choose the lesion location probability corresponding to the breast cancer probability and the lesion classification corresponding to the breast density classification probability. Probability.
  • the first computing device 120 may generate a corresponding initial first attention map for each lesion location probability, and set each initial first attention map according to the correlation between each focus location probability and the final goal to be achieved. According to the weight of the force map, multiple initial first attention maps are weighted and summed to generate the final first attention map based on the initial medical image. In some embodiments, the first computing device 120 may generate a corresponding initial second attention map for each lesion classification probability, and set each initial second attention map according to the correlation between each lesion classification probability and the final goal to be achieved. According to the weight of the force map, multiple initial second attention maps are weighted and summed to generate the final second attention map based on the initial medical image.
  • the obtained lesion classification probabilities include breast density classification probabilities and breast distortion classification probabilities, and the ultimate goal is to diagnose breast cancer. Since the correlation between breast cancer diagnosis and breast density judgment results is high, the weight of the initial second attention map obtained from the breast density classification probability can be set to a larger value, and the weight of the initial second attention map obtained from the breast distortion classification probability can be set to a larger value. The weight is set to a smaller value; based on the weights of the two initial second attention maps, the two initial second attention maps are weighted and summed to obtain the final second attention map based on the initial medical image.
  • Figure 5 is an exemplary flowchart of an image detection method according to some embodiments of this specification.
  • process 500 includes the following steps.
  • process 500 may be performed by first computing device 120 or detection result generation module 230.
  • the first computing device 120 may use the second model to generate The detection result of the target object, where the second model is a trained machine learning model.
  • Step 510 For each of the plurality of initial medical images, use the second model to process at least one of the first attention map and the second attention map, as well as the initial medical image, to obtain an initial detection result of the initial medical image.
  • the multiple initial medical images may include multiple initial medical images corresponding to multiple viewing angles, multiple initial medical images from the same viewing angle collected at different times, multiple initial medical images of different modalities, etc. .
  • each of the initial medical images may correspond to at least one attention map, wherein the attention maps may be generated based on segmented image blocks of the initial medical images, for example, the first attention map and /or second attention map.
  • the first computing device 120 may combine the initial medical image and at least one corresponding attention map (eg, the first attention map and/or the second attention map). Attention map) is input into the second model to obtain the output detection result corresponding to the initial medical image.
  • the detection results corresponding to the initial medical image may include at least one of a lesion location result and a lesion classification result.
  • the lesion localization results include the location and range of the lesions (for example, breast cancer lesions, breast calcification lesions, breast vascular calcification lesions, etc.)
  • the lesion classification results include the types of lesions (for example, benign and malignant breast, breast calcification, breast calcification, etc.) Mass properties, etc.), grading (for example, breast BI-RADS grading, etc.), etc.
  • the detection results corresponding to the initial medical image may include image type determination results of the initial medical image (for example, images of breast nipple depression type, breast gland density type images, etc.).
  • the lesion localization results and the lesion classification results can be expressed in a probabilistic form.
  • the second model may be a convolutional neural network (Convolutional Neural Networks, CNN) such as U-Net.
  • CNN convolutional Neural Networks
  • the second model may include various machine learning models, such as convolutional neural networks, fully convolutional networks (Fully Convolutional Networks, FCN) and other neural network models.
  • the second model can be obtained by unsupervised/supervised training of the initial second model (ie, the untrained second model) using medical image samples and corresponding attention map samples as training samples, where ,These medical image samples can include various types of ,medical images, for example, with lesions, without lesions, ,multiple viewing angles, etc.
  • the second model can be obtained by training the initial second model through a single algorithm, an integrated algorithm, medical image samples and corresponding attention map samples.
  • the second model can be obtained through joint training with the first model, wherein the output of the first model can be used as a training sample for the second model.
  • step 740 for more information on how to train the second model, please refer to the relevant description of step 740, which will not be described again here.
  • the first computing device 120 may determine the initial detection result of the initial medical image through other methods described in step 330, for example, through manual processing.
  • Step 520 Generate detection results of the target object based on the initial detection results of multiple initial medical images.
  • the first computing device 120 may synthesize the initial detection results of these initial medical images to generate a detection result of the target object.
  • the plurality of initial medical images may include medical images from two views, wherein the first view may be a craniocaudal (CC) view, and the second view may be a mediolateral oblique (MLO) view. view (that is, the person is lying on his side when shooting), the first computing device 120 can obtain two detection results using the images from the two viewing angles, and then average the two detection results, and use the averaged result as the target object. Test results.
  • CC craniocaudal
  • MLO mediolateral oblique
  • the first computing device 120 can perform a weighted average of the initial detection results of each initial medical image to obtain a weighted probability value and use it as a target Object detection results.
  • the weight value can be determined based on experience, historical data, etc.
  • the first computing device 120 can input multiple initial medical images and corresponding attention maps into the second model to directly obtain the output detection result of the target object. That is, the step of synthesizing the initial detection results of multiple initial medical images is integrated into the second model.
  • the detection results of the target object are obtained based on multiple medical images from multiple perspectives and corresponding attention maps, which improves the diagnostic accuracy; the diagnostic efficiency is improved by using machine learning models, while reducing the burden on doctors. .
  • only a single initial medical image may be used to generate the detection result of the target object.
  • the initial medical image and the corresponding at least one attention map can be input into the second model to obtain the detection result.
  • Figure 6 is a schematic diagram of an image detection method according to some embodiments of this specification.
  • the first computing device 120 may divide the initial medical image 610 into blocks to obtain segmented image blocks 620 , where the segmented image blocks 620 may include multiple segments.
  • the segmented image blocks 620 may include multiple segments.
  • the first computing device 120 can input each segmented image block 620 into the first model 630 to obtain the output lesion location probability 631 and the lesion classification probability 632.
  • the first computing device 120 may obtain the first attention map 650 based on the position information 640 and the lesion location probability 631 of each segmented image block 620, and/or obtain the first attention map 650 based on the position information 640 and the lesion classification probability 632 of each segmented image block 620.
  • the position information 640 is the position information of the segmented image block 620 in the initial medical image 610 .
  • the first attention map and the second attention map please refer to step 320 and the relevant description in Figure 4, which will not be described again here.
  • the first computing device 120 may input the initial medical image 610, the first attention map 650 and/or the second attention map 660 into the second model 670 to obtain the detection result 680 .
  • the detection result 680 may be the detection result corresponding to the initial medical image 610 or the detection result of the target object.
  • Figure 7 is an exemplary flow chart of a model training method according to some embodiments of this specification.
  • process 700 includes the following steps.
  • process 700 may be performed by the second computing device 130 or the model training module.
  • the second computing device 130 can jointly train the initial first model and the initial second model to obtain the trained first model and the second model.
  • Step 710 Obtain segmented image block samples of the medical image sample.
  • the second computing device 130 can acquire medical images as medical image samples through various methods, and then perform block operations on these medical image samples to obtain segmented image block samples of the medical image samples. In some embodiments, the second computing device 130 may instruct the first computing device 120 to tile the medical image samples.
  • the segmented image block samples may be sampled from medical image samples marked with lesion areas.
  • the second computing device 130 may select a random location within the boundaries of the lesion as the center of each image patch. If the obtained image block contains an insufficient number of annotated lesion pixels or the range of the area of interest (for example, breast, etc.) contained in the image block is too small, a new image block can be resampled, and then a data enhancement method (for example, , random rotation, random flipping, random scaling, etc.) to enhance the sample data set.
  • a data enhancement method for example, , random rotation, random flipping, random scaling, etc.
  • step 310 For more information on how to obtain medical images and perform block operations on the images, please refer to the relevant description of step 310, in This will not be described again.
  • Step 720 Use the segmented image block samples as training samples for the first model, train the initial first model, and obtain the first model.
  • the second computing device 130 may use the segmented image block samples as training samples for the first model, and use the lesion classification probability samples and lesion location probability samples corresponding to the segmented image block samples as training labels (through manual annotation or historical data, etc.), train the initial first model, thereby obtaining the first model.
  • the second computing device 130 may use the segmented image patch samples as input to the initial first model, and compare the predicted lesion classification probability and the predicted lesion location probability output by the initial first model with the training labels.
  • the value of the loss function can be determined based on the comparison results. For example, the first value of the segmentation loss (for example, Jaccard loss and focal loss) can be obtained according to the comparison results of the predicted lesion location probability and the lesion location probability sample, and the classification loss (for example, The second value of cross entropy loss).
  • the value of the loss function value is the weighted sum of the first value and the second value. Iteratively updates the parameter values of the first initial model according to the value of the loss function until the preset condition is met.
  • the preset conditions may include reaching a preset number of iterations, the difference between the predicted lesion classification probability and the predicted lesion location probability and the training label being less than a threshold, etc.
  • Step 730 Based on the medical image sample and the first model, generate an attention map sample corresponding to the medical image sample.
  • the second computing device 130 can generate an attention map sample corresponding to the medical image sample according to the medical image sample and the first model. Specifically, the second computing device 130 can input each segmented image block sample into the first model to obtain the output lesion classification probability and lesion location probability. Then based on the position information and lesion classification probability and/or lesion location probability of each segmented image block sample, the second computing device 130 can generate a first attention map sample and/or a second attention map sample corresponding to the medical image sample. For more information on how to generate the attention map, please refer to step 320 and the relevant description in Figure 4, which will not be described again here.
  • Step 740 Use the attention map samples and the medical image samples as training samples for the second model, train the initial second model, and obtain the second model.
  • the second computing device 130 may use the medical image samples and their corresponding attention map samples as training samples for the second model, and use the detection result samples corresponding to the medical image samples as training samples. Label (obtained through manual annotation or historical data, etc.), train the initial second model, and obtain the second model.
  • the type of the loss function used to train the initial second model may be determined according to the type of detection result output by the second model. Taking the detection result as the classification of benign and malignant breast tumors as an example, the loss function can be cross entropy (for example, binary cross entropy loss for binary classification).
  • the specific method of training the second model is similar to training the first model, and will not be described again here.
  • the first model and the second model can also be trained separately, wherein the respective medical image samples used for training by the two models may be the same or different.
  • the first model and the second model can also be jointly trained.
  • the training samples for joint training may include medical image samples and corresponding detection result samples.
  • the second computing device 130 can input the segmented image block samples of the medical image sample into the initial first model to obtain the output predicted lesion classification probability and predicted lesion location probability; based on the position information of each segmented image block sample and Predict the lesion classification probability and/or predict the lesion location probability, and generate the first attention map sample and/or the second attention map sample corresponding to the medical image sample; and then combine the medical image sample with the generated first attention map sample and/or the second attention map sample.
  • the attention map sample is input to the initial second model, and the network parameters of the initial first model and the initial second model are iteratively adjusted and set at the same time based on the prediction detection results output by the initial second model until the preset conditions are met.
  • the preset conditions may include reaching a preset number of iterations, the difference between the predicted detection result and the training label being less than a threshold, etc.
  • Figure 8 is a schematic structural diagram of a first model according to some embodiments of this specification.
  • the first model 800 may include an encoder 810, a decoder 820, and a classification framework 830.
  • the input to the first model 800 is a segmented image patch 850.
  • the encoder 810 can be used to extract the coding feature information of the segmented image block 850;
  • the decoder 820 can be used to extract the decoding feature information of the segmented image block 850 based on the coding feature information, and generate the segmented image based on the decoding feature information.
  • the lesion localization probability 860 corresponding to block 850; the classification framework 830 can be used to encode feature information based on
  • the lesion classification probability 870 corresponding to the segmented image block 850 is generated by at least a part of the decoded feature information.
  • the encoder 810 may extract encoding feature information from the segmented image blocks 850 through convolution operations, normalization, downsampling, and the like.
  • the encoding layer in the encoder 810 may include encoding layers 811-1, 811-2, 811-3, ..., 811-n, where n ⁇ 2.
  • the decoder 820 may extract the decoding feature information of the segmented image block 850 from the encoding feature information through a convolution operation, upsampling, or the like.
  • the number of upsampling may be the same as the number of downsampling.
  • the decoding layer in the decoder 820 may include decoding layers 821-1, 821-2, 821-3, ..., 821-n.
  • the decoder 820 may perform feature decoding through decoding layers, wherein an upsampling operation is included between every two decoding layers.
  • the decoder 820 can perform 4 upsampling operations.
  • the encoder 810 may input the encoded feature information obtained after each feature extraction into the decoding layer.
  • the encoding feature information obtained in the encoding layer 811-1 can be input to the decoding layer 821-2.
  • the decoder 820 of the first model 800 may include an attention block.
  • the attention block in the decoder 820 may correspond to its decoding layer one-to-one, including the attention block 822-1, 822-2, 822-3,..., 822-n.
  • the attention block can be used to enhance spatial features and channel features based on the weight of spatial features, the weight of channel features and original features.
  • the first model 800 may also include a normalization layer, which may be used to connect the encoder and the decoder and normalize the encoding feature information.
  • the normalization layer of the first model 800 includes normalization layers 840-1, 840-2, 840-3, ..., 840-n.
  • a normalization layer may be located between the encoder 810 and the decoder 820, with each normalization layer connecting the corresponding encoding layer and decoding layer.
  • the encoder 810 can input the encoded feature information obtained after each feature extraction into the corresponding normalization layer; the normalization layer normalizes the encoded feature information and converts the normalized
  • the encoding feature information is input to the corresponding decoding layer.
  • the encoding layer 811-2 inputs the encoded feature information obtained by feature extraction into the normalization layer 840-2.
  • the normalization layer 840-2 normalizes the encoded feature information and inputs the processed information to the decoding layer. 821-2 in.
  • the normalization operation may include normalizing encoded feature information (eg, encoder feature map) at different scales.
  • the decoder 820 can fuse the final coding feature information of the segmented image block 850 obtained after multiple upsamplings, thereby obtaining the lesion location probability 860 corresponding to the segmented image block 850.
  • Possible beneficial effects brought about by the embodiments of this specification include but are not limited to: (1) By segmenting medical images (for example, mammography images, etc.) into image blocks, and inputting them into a machine learning model to obtain lesion location probability and lesion classification probability , and further obtain the diagnosis results of the disease through the machine learning model, so that the lesions (such as breast lesions, etc.) can be accurately located, avoiding false positive detection results; moreover, it can combine accurately positioned lesions and medical images, It provides reasonable disease auxiliary diagnosis results, provides doctors with objective standards, reduces the influence of subjective factors, and thus improves diagnostic accuracy; by using machine learning models, it reduces doctor workload to a certain extent, improves diagnosis efficiency, and avoids Eliminate unnecessary additional examinations for patients; (2) By using a multi-task machine learning model, the lesion location probability and the lesion classification probability can be output simultaneously, thereby improving the convenience of using the model, and at the same time, accurate detection can be obtained based on multiple detection results.
  • medical images for example, mammography images,
  • Diagnosis results (3)
  • the detection results (lesion location probability and lesion classification probability) and diagnosis results obtained through the machine learning model can include multiple types, thus improving the adaptability of the model and well meeting the diverse diagnostic needs.
  • different embodiments may produce different beneficial effects.
  • the possible beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.
  • numbers are used to describe the quantities of components and properties. It should be understood that such numbers used to describe the embodiments are modified by the modifiers "about”, “approximately” or “substantially” in some examples. Grooming. Unless otherwise stated, “about,” “approximately,” or “substantially” means that the stated number is allowed to vary by ⁇ 20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximations that may vary depending on the desired features of the individual embodiment. In some embodiments, numerical parameters should account for the specified number of significant digits and use general digit preservation methods. Although the numerical ranges and parameters used to identify the breadth of ranges in some embodiments of this specification are approximations, in specific embodiments, such numerical values are set as accurately as is feasible.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

一种图像检测方法、系统、装置及存储介质，该方法包括：获取目标对象的初始医学图像中的多个分割图像块（310）；基于多个分割图像块，生成初始医学图像的至少一张注意力图（320），其中，至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，第一注意力图与病灶定位相关，第二注意力图与病灶分类相关；基于至少一张注意力图，生成目标对象的检测结果（330）。

Description

图像检测方法、系统、装置及存储介质
相关申请的交叉引用
本申请要求2022年04月14日提交的名称为“图像检测方法、装置及计算机设备”的中国专利申请202210389475.9的优先权，上述申请的全部内容以引用方式被完全包含在此。
技术领域
本说明书涉及医学图像处理领域，特别涉及一种图像检测方法、系统、装置及存储介质。
背景技术
医学影像通常用于对疾病进行筛查和诊断，尤其是各种癌症的早期排查和诊断。通常采用人工或计算机辅助检测的方式来判断病灶以及癌症的发展程度(例如，良恶性、早晚期等)。以乳腺癌为例，钼靶筛查是乳腺癌筛查的常用方式，临床上需要医生基于钼靶图像的影像征象(例如，肿块和钙化的性质、形态、乳头是否凹陷等)来人工判断乳腺肿瘤的良恶性。但是，依靠影像征象判断缺乏定量标准、不同医生的判断标准存在不一致性，并且依靠影像征象诊断存在病灶检测敏感性和特异性不高等问题。而当前的计算机辅助检测的效果较差，导致癌症检测的准确性降低，医生需要复查计算机辅助检查系统中标记的假阳性标签，而病人则需要接受额外的活检复查。这些大大增加了放射科医生的工作量，同时增加了对病人不必要的额外检查。
因此，希望提供一种图像检测方法以提高癌症等疾病的检测准确率。
发明内容
本说明书实施例之一提供一种图像检测方法。所述方法包括：获取目标对象的初始医学图像中的多个分割图像块；基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，所述至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，所述第一注意力图与病灶定位相关，所述第二注意力图与病灶分类相关；基于所述至少一张注意力图，生成所述目标对象的检测结果。
在一些实施例中，所述第一注意力图包括所述初始医学图像中的元素的第一注意力值，所述第一注意力值与所述元素属于病灶的概率相关，所述第二注意力图包括所述初始医学图像中的元素的第二注意力值，所述第二注意力值与所述元素的病灶分类相关。
在一些实施例中，可以对所述多个分割图像块中的每个分割图像块，使用第一模型生成所述分割图像块的病灶定位概率和病灶分类概率，其中，所述第一模型为训练好的机器学习模型；基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图。
在一些实施例中，可以基于所述每个分割图像块的位置信息和所述病灶定位概率生成所述第一注意力图；基于所述每个分割图像块的所述位置信息和所述病灶分类概率生成所述第二注意力图。
在一些实施例中，可以将所述病灶定位概率和所述病灶分类概率重新映射到所述初始医学图像，得到所述第一注意力图和所述第二注意力图。其中，所述映射可以包括线性映射和高斯映射中的至少一种。
在一些实施例中，可以获取所述多个分割图像块中的每个分割图像块对应的权重，其中，所述第一注意力图和所述第二注意力图可以进一步基于所述权重生成。
在一些实施例中，所述多个分割图像块至少可以包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块，所述至少一个第一分割图像块可以用于生成所述病灶定位概率，所述至少一个第二分割图像块可以用于生成所述病灶分类概率。
在一些实施例中，所述第一模型可以包括编码器、解码器和分类框架。其中，对所述每个分割图像块，所述编码器可以用于提取所述分割图像块的编码特征信息；所述解码器可以用于基于所述编码特征信息提取所述分割图像块的解码特征信息，并基于所述解码特征信息生成所述分割图像块的所述病灶定位概率；所述分类框架可以用于基于所述编码特征信息和所述解码特征信息中的至少一部分生成所述分割图像块的所述病灶分类概率。
在一些实施例中，所述第一模型还可以包括归一化层，其中，所述归一化层可以用于连接所述编码器和所述解码器，并对所述编码特征信息进行归一化处理。
在一些实施例中，所述解码器可以包括注意力块，所述注意力块用于基于空间特征的权重、通道特征的权重和原始特征，对所述空间特征和所述通道特征进行加强处理。
在一些实施例中，可以基于所述至少一张注意力图和所述初始医学图像，使用第二模型生成所述目标对象的检测结果，其中，所述第二模型为训练好的机器学习模型。
在一些实施例中，所述初始医学图像可以包括对应多个视角的多张初始医学图像，所述至少一张注意力图可以包括每张所述初始医学图像对应的至少一张注意力图。
在一些实施例中，对所述多张初始医学图像中的每一张，可以利用所述第二模型处理所述初始医学图像及其对应的至少一张注意力图，得到所述初始医学图像的初始检测结果；基于所述多张初始医学图像的初始检测结果，生成所述目标对象的检测结果。
在一些实施例中，所述至少一张注意力图可以通过第一模型获得，所述目标对象的检测结果可以通过第二模型获得，所述第一模型和所述第二模型可以通过以下方式训练获得：获取医学图像样本的分割图像块样本；将所述分割图像块样本作为所述第一模型的训练样本，训练初始第一模型，得到所述第一模型；基于所述医学图像样本和所述第一模型，生成所述医学图像样本对应的注意力图样本；将所述注意力图样本和所述医学图像样本作为所述第二模型的训练样本，训练初始第二模型，得到所述第二模型。
本说明书实施例之一提供一种图像检测系统，包括图像块获取模块、注意力图生成模块和检测结果生成模块；所述图像块获取模块用于获取目标对象的初始医学图像中的多个分割图像块；所述注意力图生成模块用于基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，所述至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，所述第一注意力图与病灶定位相关，所述第二注意力图与病灶分类相关；所述检测结果生成模块用于基于所述至少一张注意力图，生成所述目标对象的检测结果。
本说明书实施例之一提供一种图像检测系统，包括至少一个存储器，所述存储器存储至少一组计算机指令；以及至少一个处理器，所述处理器与所述存储器通信，当所述至少一组计算机指令被执行时，所述处理器执行以下方法；获取目标对象的初始医学图像中的多个分割图像块；基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，所述至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，所述第一注意力图与病灶定位相关，所述第二注意力图与病灶分类相关；基于所述至少一张注意力图，生成所述目标对象的检测结果。
本说明书实施例之一提供一种图像检测装置，包括处理器，所述处理器用于执行所述图像检测方法。
本说明书实施例之一提供一种计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行所述图像检测方法。
附图说明
本说明书将以示例性实施例的方式进一步说明，这些示例性实施例将通过附图进行详细描述。这些实施例并非限制性的，在这些实施例中，相同的编号表示相同的结构，其中：
图1是根据本说明书一些实施例所示的图像检测系统的应用场景示意图；
图2是根据本说明书一些实施例所示的图像检测系统的示意图；
图3是根据本说明书一些实施例所示的图像检测方法的示例性流程图；
图4是根据本说明书一些实施例所示的图像检测方法的示例性流程图；
图5是根据本说明书一些实施例所示的图像检测方法的示例性流程图；
图6是根据本说明书一些实施例所示的图像检测方法的示意图；
图7是根据本说明书一些实施例所示的模型训练方法的示例性流程图；
图8是根据本说明书一些实施例所示的第一模型的结构示意图；
图9是根据本说明书一些实施例所示的初始医学图像的示意图。
具体实施方式
为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本说明书的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本说明书应用于其它类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。
应当理解，本文使用的“系统”、“装置”、“单元”和/或“模块”是用于区分不同级别的不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。
如本说明书和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其它的步骤或元素。
本说明书中使用了流程图用来说明根据本说明书的实施例的系统所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
图1是根据本说明书一些实施例所示的图像检测系统的应用场景示意图。
如图1所示，在一些实施例中，图像检测系统100(以下称为系统100)可以包括医学影像设备110、第一计算设备120、第二计算设备130、用户终端140、存储设备150和网络160。
医学影像设备110可以指利用不同的媒介，将目标物体(例如人体)内部的结构重现为影像的装置。在一些实施例中，医学影像设备110可以是任何可以对目标物体(例如人体)的指定身体部位进行成像或治疗的设备，例如，钼靶乳腺X射线机、MRI(Magnetic Resonance Imaging)、CT(Computed Tomography)、PET(Positron Emission Tomography)、超声诊断设备等。上面提供的医学影像设备110仅用于说明目的，而非对其范围的限制。在一些实施例中，医学影像设备110可以获取患者的指定部位(例如，乳腺、脑部、胃部等)的医学图像(例如，磁共振(MRI)图像、CT图像、超声图像、钼靶图像等)并发送至系统100的其它组件(例如，第一计算设备120、第二计算设备130、存储设备150)。在一些实施例中，医学影像设备110可以通过网络160与系统100中的其它组件进行数据和/或信息的交换。
第一计算设备120和第二计算设备130是具有计算和处理能力的系统，可以包括各种计算机，比如服务器、个人计算机，也可以是由多台计算机以各种结构连接组成的计算平台。在一些实施例中，第一计算设备120与第二计算设备130可以是同一个设备，也可以是不同的设备。
第一计算设备120和第二计算设备130中可以包括一个或多个子处理设备(例如，单核处理设备或多核多芯处理设备)，处理设备可以执行程序指令。仅作为示例，处理设备可以包括各种常见的通用中央处理器(central processing unit，CPU)，图形处理器(Graphics Processing Unit，GPU)，微处理器，特殊应用集成电路(application-specific integrated circuit，ASIC)，或其它类型的集成电路。
第一计算设备120可以处理与医学图像相关的信息和数据。在一些实施例中，第一计算设备120可以执行如本说明书一些实施例所示的图像检测方法，得到至少一种图像检测结果，即目标对象(例如，病人等)的检测结果。示例性的检测结果可以包括乳腺良恶性诊断结果、乳腺BI-RADS诊断结果、乳腺密度诊断结果、乳腺钙化诊断结果、乳腺肿块性质诊断结果等的其中至少一种。在一些实施例中，第一计算设备120可以通过机器学习模型得到疾病的检测结果(例如，病灶定位概率图和病灶分类概率图)。在一些实施例中，第一计算设备120可以从第二计算设备130获取训练好的机器学习模型。在一些实施例中，第一计算设备120可以基于病灶定位概率图和病灶分类概率图确定目标对象的检测结果，即疾病的诊断结果。在一些实施例中，第一计算设备120可以通过网络160和/或系统100中的其它组件(例如，医学影像设备110、第二计算设备130、用户终端140、存储设备150)交换信息和数据。在一些实施例中，第一计算设备120可以直接与第二计算设备130连接并交换信息和/或数据。
第二计算设备130可以用于模型训练。在一些实施例中，第二计算设备130可以执行如本说明书一些实施例所示的机器学习模型的训练方法，得到训练好的机器学习模型。在一些实施例中，第二计算设备130可以从医学影像设备110获取图像信息作为模型的训练数据。在一些实施例中，第一计算设备120 和第二计算设备130也可以是同一个计算设备。
用户终端140可以接收和/或展示医学图像的处理结果。在一些实施例中，用户终端140可以从第一计算设备120接收医学图像的图像检测结果，基于此图像检测结果对患者进行诊断和治疗。在一些实施例中，用户终端140可以通过指令使第一计算设备120执行如本说明书一些实施例所示的图像检测方法。在一些实施例中，用户终端140可以控制医学影像设备110以获取特定部位的医学图像。在一些实施例中，用户终端140可以是移动设备140-1、平板计算机140-2、膝上型计算机140-3、台式计算机等其它具有输入和/或输出功能的设备中的一种或其任意组合。
存储设备150可以存储其它设备产生的数据或信息。在一些实施例中，存储设备150可以存储医学影像设备110采集的医学图像。在一些实施例中，存储设备150可以存储第一计算设备120和/或第二计算设备130处理后的数据和/或信息，例如，病灶定位概率图、病灶分类概率图、注意力图等。存储设备150可以包括一个或多个存储组件，每个存储组件可以是一个独立的设备，也可以是其它设备的一部分。存储设备可以是本地的，也可以通过云实现。
网络160可以连接系统的各组成部分和/或连接系统与外部资源部分。网络160使得各组成部分之间，以及与系统之外其它部分之间可以进行通讯，促进数据和/或信息的交换。在一些实施例中，系统100中的一个或多个组件(例如，医学影像设备110、第一计算设备120、第二计算设备130、用户终端140、存储设备150)可通过网络160发送数据和/或信息给其它组件。在一些实施例中，网络160可以是有线网络或无线网络中的任意一种或多种。
应该注意的是，上述描述仅出于说明性目的而提供，并不旨在限制本说明书的范围。对于本领域普通技术人员而言，在本说明书内容的指导下，可做出多种变化和修改。可以以各种方式组合本说明书描述的示例性实施例的特征、结构、方法和其它特征，以获得另外的和/或替代的示例性实施例。例如，第一计算设备120和/或第二计算设备130可以是基于云计算平台的，例如公共云、私有云、社区和混合云等。然而，这些变化与修改不会背离本说明书的范围。
图2是根据本说明书一些实施例所示的图像检测系统的示意图。
如图2所示，在一些实施例中，图像检测系统200可以包括图像块获取模块210、注意力图生成模块220和检测结果生成模块230。
在一些实施例中，图像块获取模块210可以用于获取目标对象的初始医学图像中的多个分割图像块。
在一些实施例中，注意力图生成模块220可以用于基于多个分割图像块，生成初始医学图像的至少一张注意力图。其中，至少一张注意力图可以包括第一注意力图和第二注意力图中的至少一个，第一注意力图与病灶定位相关，第二注意力图与病灶分类相关。
在一些实施例中，第一注意力图可以包括初始医学图像中的元素的第一注意力值，第一注意力值可以与元素属于病灶的概率相关，第二注意力图可以包括初始医学图像中的元素的第二注意力值，第二注意力值可以与元素的病灶分类相关。
在一些实施例中，对多个分割图像块中的每个分割图像块，注意力图生成模块220可以使用第一模型生成分割图像块的病灶定位概率和病灶分类概率，其中，第一模型可以为训练好的机器学习模型。
在一些实施例中，第一模型可以包括编码器、解码器和分类框架。其中，对每个分割图像块，编码器可以用于提取分割图像块的编码特征信息；解码器可以用于基于编码特征信息提取分割图像块的解码特征信息，并基于解码特征信息生成分割图像块的病灶定位概率；分类框架可以用于基于编码特征信息和解码特征信息中的至少一部分生成分割图像块的病灶分类概率。
在一些实施例中，第一模型还可以包括归一化层。其中，归一化层可以用于连接编码器和解码器，并对编码特征信息进行归一化处理。
在一些实施例中，解码器可以包括注意力块。其中，注意力块可以用于基于空间特征的权重、通道特征的权重和原始特征，对空间特征和通道特征进行加强处理。
在一些实施例中，注意力图生成模块220可以基于病灶定位概率和病灶分类概率得到第一注意力图和第二注意力图。
在一些实施例中，注意力图生成模块220可以基于每个分割图像块的位置信息和病灶定位概率生成第一注意力图；基于每个分割图像块的位置信息和病灶分类概率生成第二注意力图。
在一些实施例中，对多个分割图像块中的每个分割图像块，注意力图生成模块220可以获取分割图像块所对应的权重，其中，第一注意力图和第二注意力图可以进一步基于权重生成。
在一些实施例中，多个分割图像块至少可以包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块，至少一个第一分割块可以用于生成病灶定位概率，至少一个第二分割图像块可以用于生成病灶分类概率。
在一些实施例中，注意力图生成模块220可以将病灶定位概率和病灶分类概率重新映射到初始医学图像，得到第一注意力图和第二注意力图，其中，映射可以包括线性映射和高斯映射中的至少一种。
在一些实施例中，检测结果生成模块230可以用于基于至少一张注意力图，生成目标对象的检测结果。
在一些实施例中，检测结果生成模块230可以基于至少一张注意力图和初始医学图像，使用第二模型生成目标对象的检测结果，其中，第二模型可以为训练好的机器学习模型。
在一些实施例中，初始医学图像可以包括对应多个视角的多张初始医学图像，至少一张注意力图包括可以每张初始医学图像对应的至少一张注意力图。对这些初始医学图像中的每一张，检测结果生成模块230可以利用第二模型处理该初始医学图像及其对应的至少一张注意力图，得到该初始医学图像的初始检测结果。检测结果生成模块230可以基于这些初始医学图像的初始检测结果，生成目标对象的检测结果。
在一些实施例中，至少一张注意力图可以通过第一模型获得，目标对象的检测结果可以通过第二模型获得。在一些实施例中，图像检测系统200还可以包括模型训练模块(图中未示出)。模型训练模块可以获取医学图像样本的分割图像块样本；将分割图像块样本作为所述第一模型的训练样本，训练初始第一模型，得到第一模型；基于医学图像样本和第一模型，生成医学图像样本对应的注意力图样本；将注意力图样本和医学图像样本作为第二模型的训练样本，训练初始第二模型，得到第二模型。
在一些实施例中，图像检测系统200可以由第一计算设备120和/或第二计算设备130实现。例如，图像块获取模块210、注意力图生成模块220和检测结果生成模块230可以由第一计算设备120实现。又例如，模型训练模块可以由第二计算设备130实现。
图3是根据本说明书一些实施例所示的图像检测方法的示例性流程图。
如图3所示，流程300包括下述步骤。在一些实施例中，流程300可以由第一计算设备120执行。
步骤310，获取目标对象的初始医学图像中的多个分割图像块。在一些实施例中，步骤310可以由图像块获取模块210执行。
目标对象是需要获取检查结果的对象，例如，已确诊的病人、进行癌症筛查的检查对象等。在一些实施例中，目标对象可以包括人体的特定部位，例如，乳腺、脑部、胃部等。在一些实施例中，目标对象可以包括模拟的对象，例如，由计算机模拟得到的数字人体等。
初始医学图像是用于获取目标对象的检测结果的原始医学图像，例如，MRI图像、CT图像、超声图像、钼靶图像等。在一些实施例中，初始医学图像可以通过医学影像设备(例如，医学影像设备110)扫描目标对象获取。在一些实施例中，初始医学图像可以通过其它方式获取，例如，从存储设备获取、对数字人体进行模拟扫描获取等。
在一些实施例中，第一计算设备120可以对初始医学图像进行初步筛选，去除其中不符合质量要求(例如，清晰度不足、不包含感兴趣区域等)的图像。
在一些实施例中，若初始医学图像长宽比大于或等于预设值(例如，10:1等)，可以认为图像过于细长，第一计算设备120可以对初始医学图像使用填充(padding)等方法将旁边补零，以满足长宽比要求。
分割图像块(简称为图像块)是指医学图像经过分割/分块得到的图像的分块。医学影像设备采集的初始医学图像一般为大尺寸图像，图像中的信息量较大，若直接通过初始医学图像进行医学分析，不仅会增加计算设备处理数据的压力，还会由于图像的信息量过大不能进行针对性的分析，导致分析结果可能不准确。在一些实施例中，第一计算设备120可以通过各种方式(例如，滑窗法、随机采样法等)对初始 医学图像进行分块，得到多个分割图像块。在一些实施例中，第一计算设备120可以从其他设备(例如，其他计算设备、存储设备等)获取初始医学图像的多个分割图像块。
初始医学图像中，除感兴趣区域(例如，乳腺、脑部、胃部等)外，还可能包括与感兴趣区域无关的无用信息(例如，背景、其他身体部位等)，这会导致计算设备处理数据时存在大量无效工作量，从而影响分析结果。另外，由于图像对比度等原因，可能会导致感兴趣区域与图像其他部分难以区分，从而影响后续的处理。在一些实施例中，在将初始医学图像分块之前，第一计算设备120可以基于初始医学图像中的图像信息(例如，像素信息/灰度信息、RGB信息、深度信息等)将感兴趣区域对应的部分分割出来，得到感兴趣区域对应的初始医学图像。
图9为根据本说明书一些实施例所示的初始医学图像的示意图，其中，感兴趣区域为乳腺。图9中左侧图像为未经过处理的初始医学图像，可以看出，其中包括大量黑色的背景区域，属于无用信息。如果该背景区域也参加分块，则会导致大量无效工作量，且乳房轮廓与周围其他区域边界不明显，难以区分。图9中右侧图像为将乳房分割出来得到的初始医学图像，可以看出，大量无用信息已被分割出去，乳房轮廓明显，边界清晰。为方便描述，扫描得到的初始医学图像和对初始医学图像进行感兴趣区域分割后得到的医学图像在后文统称为初始医学图像。
在一些实施例中，第一计算设备120可以基于图像信息将初始医学图像进行分块。例如，基于初始医学图像的像素按照预设的图像块尺寸进行划分，即将其划分为预设大小的图像块。又例如，基于初始医学图像的RGB信息按照预设的RGB范围进行划分，划分为不同RGB范围的图像块。又例如，基于初始医学图像的深度信息按照预设的深度范围进行划分，划分为不同深度范围的图像块。在一些实施例中，第一计算设备120可以通过其他方式将初始医学图像进行分块。例如，通过检测框在初始医学图像中进行图像标定，获得多个图像检测框等。
在一些实施例中，第一计算设备120可以将初始医学图像分为各种大小的图像块，例如，固定大小的图像块、多种大小图像块的组合等。在一些实施例中，图像块可以包括二维图像块(例如，尺寸为512*512)、三维图像块(例如，尺寸为512*512*512)等。在一些实施例中，第一计算设备120可以从初始医学图像的所有分割图像块中选取预设数量(例如，125、225等)的分割图像块，以进行进一步处理。仅作为示例，假设初始医学图像的大小为512*512，则可以按照以下方式其中之一进行分块：分割成大小为64*64的图像块，随机选取125个大小为64*64的分割图像块；分割成大小为64*64的图像块以及分割成大小为32*32的图像块，随机选取125个大小为64*64的图像块以及125个大小为32*32的图像块。
对于病灶尺寸相对于初始医学图像较小的情况，通常，图像分割后每一个图像块的尺寸较小则有利于后续的分析，但是，图像块尺寸越小，分割后得到的分割图像块的数量就越多，这样会加重计算设备的分析压力，导致图像分析效率的降低。所以，在一些实施例中，分块策略可以通过各种方式确定，例如，根据分析需求以及初始医学图像的实际情况设置不同的分块策略，以对不同的初始医学图像进行不同的处理，以此来提高计算设备的分析效率，同时保证分析的质量。又例如，若初始医学图像存在异常的概率不足10％，则对该初始医学图像进行分块处理时，第一计算设备120可以设置较大的图像块大小，这样分割得到的图像块的数量较少，便于计算设备快速地对分割图像块进行处理。
在一些实施例中，第一计算设备120可以基于分割图像块的大小选择对应的模型(例如，第一模型、第二模型)，以基于分割图像块执行后续处理。例如，图像块大小为64*64，就选择输入图像大小是64*64的模型进行处理。
在一些实施例中，第一计算设备120可以对初始医学图像进行多次分割，每次分割生成特定大小的图像块，以用于后续的分析，其中，每次分割的对象均为未分割前的初始医学图像。例如，可以将不同大小的图像块分别用于病灶定位操作(简称为病灶定位)和病灶分类操作(简称为病灶分类)。又例如，对于病灶定位和病灶分类中的任意一个，都可以使用不同大小的图像块进行多次操作。
在一些实施例中，第一计算设备120可以生成多个第一大小(例如，512*512等)的图像块以及多个第二大小(例如，256*256等)的图像块，第一大小的图像块用于进行后续的病灶定位(例如，生成病灶定位概率)，第二大小的图像块用于进行后续的病灶分类(例如，生成病灶分类概率)。这样通过使用较大尺寸的图像块进行病灶定位，使用较小尺寸的图像块进行病灶分类，可以得到更好的病灶定位效果 和病灶分类效果。
在一些实施例中，第一计算设备120可以使用不同大小的分割图像块来多次执行同一个操作(例如，病灶定位、病灶分类等中的任意一种)，然后将得到的多个结果综合起来得到该操作的结果。例如，对于病灶定位，可以用第一大小(例如，512*512等)的图像块进行一次操作，再用第二大小(例如，256*256等)的图像块进行一次操作，然后把两次的病灶定位结果综合起来。因为不同尺度的图像块可以提供不同的尺度信息，这样操作可以提高病灶定位结果和病灶分类结果的准确性。在一些实施例中，综合不同大小的图像块对应的病灶定位结果时，第一计算设备120可以设置对应的权重，该权重与图像块的大小提供的识别结果的准确性相关。例如，512*512大小的图像块进行病灶定位的效果更好，其权重更大，256*256大小的图像块的权重较小。
在一些实施例中，相邻分割图像块之间可以交叠。例如，相邻图像块之间交叠面积占每个图像块的总面积的25％。
步骤320，基于多个分割图像块，生成初始医学图像的至少一张注意力图。其中，至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，第一注意力图与病灶定位相关，第二注意力图与病灶分类相关。在一些实施例中，步骤320可以由注意力图生成模块220执行。
第一注意力图和第二注意力图是用于标识目标对象的疾病分析结果的数据，例如，标识了属于病灶的概率的图像、标识了病灶类型概率的图像等。在一些实施例中，第一注意力图可以包括初始医学图像中的元素(例如，像素、体素等)的第一注意力值，其中，第一注意力值可以用于病灶定位，其与元素属于病灶的概率相关。例如，第一注意力值可以与乳腺癌症的概率、乳腺肿块的概率、乳腺钙化的概率、乳腺血管钙化的概率等相关。在一些实施例中，第二注意力图可以包括初始医学图像中的元素的第二注意力值，其中，第二注意力值可以用于病灶分类，其与元素的病灶分类相关。例如，第二注意力值可以与乳腺的良恶性分类概率、乳腺的钙化性质的分类概率、乳腺肿块性质的分类概率、乳腺淋巴结性质的分类概率、乳腺软组织结构性质的分类概率等相关。在一些实施例中，第一注意力图和第二注意力图可以用多维数值(例如，伪色彩图像等)来表示。
在一些实施例中，在得到初始医学图像的多个分割图像块后，第一计算设备120可以通过第一模型处理这些分割图像块，以得到第一注意力图和/或第二注意力图。关于如何通过第一模型得到第一注意力图和/或第二注意力图的更多内容，可以参加图4的相关描述，在此不再赘述。
在一些实施例中，第一计算设备120可以通过其他方式得到第一注意力图和/或第二注意力图，例如，通过人工标注等方式。
步骤330，基于至少一张注意力图，生成目标对象的检测结果。在一些实施例中，步骤330可以由检测结果生成模块230执行。
目标对象的检测结果是指根据初始医学图像得到的对目标对象的病症的诊断结果，例如，癌症的良性/恶性判别结果、肿块性质、骨折类型等。以检测乳腺癌为例，检测结果可以包括对乳腺良恶性的诊断结果、对乳腺密度的诊断结果、对乳腺的BI-RADS分级诊断结果、对乳腺钙化的诊断结果、对乳腺肿块性质的诊断结果等中的任意一种。在一些实施例中，检测结果可以包括针对初始医学图像的图像类型的确定结果。例如，检测结果为初始医学图像的图像类型为第一类型，第一类型表征初始医学图像为乳腺乳头凹陷类型的图像。又例如，初始医学图像的图像类型为第二类型，第二类型表征初始医学图像为高乳腺腺体密度类型的图像。
在一些实施例中，在得到至少一张注意力图之后，第一计算设备120可以通过第二模型处理这些注意力图，以得到目标对象的检测结果。关于如何确定目标对象的检测结果的更多内容，可以参见图5的相关描述，在此不再赘述。
在一些实施例中，第一计算设备120可以基于注意力图中元素对应的病灶定位概率和/或病灶分类概率来确定目标对象的检测结果。例如，如果检测结果为目标对象是否有肿瘤，而第一注意力图中包含每个元素属于肿瘤病灶的概率(病灶定位概率)，则第一计算设备120可以统计概率超过阈值(例如，60％)的元素数量。如果数量超过阈值(例如，10000个)，则确定目标对象有肿瘤。又例如，如果检测结果为目标对象的肿瘤为良性还是恶性，而第二注意力图中包含每个元素属于恶性肿瘤的概率(病灶分类概率)， 则第一计算设备120可以统计恶性概率超过阈值(例如，80％)的元素数量。如果数量超过阈值(例如，5000个)，则确定目标对象的肿瘤为恶性。又例如，如果检测结果为目标对象是否患有乳腺癌，则第一计算设备120可以基于第一注意力图中的乳腺癌概率和第二注意图中的乳腺密度概率，确定目标对象是否患有乳腺癌。
在一些实施例中，第一计算设备120可以通过其他方式得到目标对象的检测结果，例如，通过人工处理注意力图等方式。
图4是根据本说明书一些实施例所示的图像检测方法的示例性流程图。
如图4所示，流程400包括下述步骤。在一些实施例中，流程400可以由第一计算设备120或注意力图生成模块220执行。在一些实施例中，通过执行流程400所示的步骤，第一计算设备120可以基于多个分割图像块，生成初始医学图像的至少一张注意力图。
步骤410，对多个分割图像块中的每个分割图像块，使用第一模型生成分割图像块的病灶定位概率和病灶分类概率。
在一些实施例中，对多个分割图像块中的每个分割图像块，第一计算设备120可以将该分割图像块输入第一模型，得到输出的该分割图像块的病灶定位概率和病灶分类概率，其中，第一模型为训练好的机器学习模型。在一些实施例中，第一模型的输出可以包括该分割图像块中每个元素(例如，像素、体素等)的病灶定位概率和病灶分类概率。
病灶定位概率(或称为病灶定位结果)表示对象(例如，图像块、图像块中的元素等)属于某种病灶(例如，腺癌症、乳腺肿块、乳腺钙化、乳腺血管钙化等)的概率。在一些实施例中，分割图像块的病灶定位概率可以由该图像块中元素的病灶定位概率组成。在一些实施例中，病灶定位概率可以包括各种表达形式，例如，数字、表格、数据库、概率图像等。在一些实施例中，病灶定位概率可以用百分比、等级等来表示，例如，20％、I级(概率小于10％)、C级(概率大于或等于60％，且小于80％)等。仅作为示例，第一模型对多个分割图像块分析后，输出第一分割图像块中元素A的病灶定位概率为10％，第二分割图像块中元素B的病灶定位概率为60％，则可以表示，第一分割图像块中元素A属于病灶的概率为10％，第二分割图像块中元素B属于病灶的概率为60％。在一些实施例中，病灶定位概率可以包括各种类型，以乳腺疾病为例，可以包括乳腺癌症的概率、乳腺肿块的概率、乳腺钙化的概率、乳腺血管钙化的概率等的其中至少一种。在一些实施例中，病灶定位概率可以由病灶定位概率图来表示。其中，不同类型的病灶定位概率可以对应不同的病灶定位概率图。
病灶分类概率(或称为病理分类结果)表示对象(例如，图像块、图像块中的元素等)属于特定类型病灶的概率，例如，乳腺肿块的良恶性概率、乳腺肿块为肿瘤/非肿瘤的概率等。在一些实施例中，分割图像块的病灶分类概率可以由该图像块中元素的病灶分类概率组成。在一些实施例中，病灶分类概率可以包括各种表达形式，例如，数字、表格、数据库、概率图像等。在一些实施例中，病灶分类概率可以用百分比、等级等来表示。仅作为示例，第一模型对多个分割图像块分析后，输出第一分割图像块中元素A为肿瘤的概率为90％，良性概率为70％；第二分割图像块中元素B为肿瘤的概率为30％，良性概率为90％。在一些实施例中，病灶分类概率可以包括各种类型。以乳腺疾病为例，可以包括乳腺的良恶性分类概率、乳腺的钙化性质的分类概率、乳腺肿块性质的分类概率、乳腺淋巴结性质的分类概率、乳腺软组织结构性质的分类概率等的其中至少一种。在一些实施例中，病灶分类概率可以由病灶分类概率图来表示，其中，不同类型的病灶分类概率可以对应不同的病灶分类概率图。在一些实施例中，病灶类型可以包括各种类型，以乳腺疾病为例，可以包括是否属于病灶、乳腺凹凸类型、乳腺腺体密度类型、乳腺BI-RADS级别类型等的其中至少一种。本说明书以下部分如无特别说明，将以病灶定位概率图表示病灶定位概率，以病灶分类概率图表示病灶分类概率，仅用于说明，不作为限定。
在一些实施例中，病灶定位概率图和病灶分类概率图可以使用多维数值(例如，伪色彩图像等)来表示。
在一些实施例中，第一模型可以使用将分割图像块与有病灶的图像块和/或无病灶的图像块进行比对的方式，来获得病灶定位概率。在一些实施例中，第一模型可以使用将分割图像块与有特定类型疾病的图像块进行比对的方式，来获得病灶分类概率和/或病灶类型。
在一些实施例中，多个分割图像块至少可以包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块。其中，至少一个第一分割块可以用于生成病灶定位概率，至少一个第二分割图像块可以用于生成病灶分类概率。
在一些实施例中，第一大小和第二大小可以相同，即第一分割图像块和第二分割图像块的大小相同。例如，用于生成病灶定位概率的第一分割图像块和用于生成病灶分类概率的第二分割图像块的大小均为512*512。
在一些实施例中，第一大小和第二大小可以不同，即第一分割图像块和第二分割图像块的大小不同。例如，用于生成病灶定位概率的第一分割图像块的第一大小可以为512*512，用于生成病灶分类概率的第二分割图像块的第二大小可以为256*256。在一些实施例中，如果第一大小和第二大小不同，第一模型可以包括至少两个不同的分支或单任务模型，以分别处理不同大小的图像块。其中，一个分支/单任务模型可以用于处理第一大小的图像块，另一个分支/单任务模型可以用于处理第二大小的图像块。在一些实施例中，第一模型可以包括多个层(layer)，其中，每个层处理的图像块大小可以不同，这些层之间可以通过降采样或升采样等操作进行连接。
在一些实施例中，多个分割图像块至少可以包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块。其中，病灶定位概率和病灶分类概率中的至少一个可以同时基于至少一个第一分割图像块和至少一个第二分割图像块生成。例如，病灶定位概率和病灶分类概率中的任意一个都基于512*512的第一分割图像块和256*256的第二分割图像块生成。关于如何基于不同大小的分割图像块生成病灶定位概率和病灶分类概率的更多内容，可以参见步骤310中相关描述，在此不再赘述。
在一些实施例中，第一模型可以为U-Net等卷积神经网络(Convolutional Neural Networks,CNN)。在一些实施例中，第一模型可以包括各种机器学习模型，例如，卷积神经网络、全卷积网络(Fully Convolutional Networks，FCN)等神经网络模型。在一些实施例中，第一模型可以为多任务模型，可以同时输出多个处理结果，例如，同时输出病灶定位概率和病灶分类概率。在一些实施例中，第一模型可以由两个单任务模型组成，其中，一个模型用于输出病灶定位结果(例如，病灶定位概率)，另一个模型用于输出病灶分类结果(例如，病灶分类概率)，两个模型的输入相同。在一些实施例中，第一模型的结构可以如图8中的第一模型800所示，详细内容可以参见图8的相关描述，在此不再赘述。
在一些实施例中，第一模型可以通过包含有病灶的图像块以及无病灶的图像块对初始第一模型(即未经过训练的第一模型)进行无监督/有监督的训练获得。在一些实施例中，第一模型可以通过单一算法、集成算法、有病灶的图像块、无病灶的图像块对初始第一模型进行训练获得。关于如何训练得到第一模型的更多内容，可以参见步骤720的相关描述，在此不再赘述。
在一些实施例中，在得到病灶定位概率和病灶分类概率之后，第一计算设备120可以基于病灶定位概率和病灶分类概率得到第一注意力图和第二注意力图。
在一些实施例中，第一计算设备120可以将病灶定位概率和病灶分类概率重新映射到初始医学图像，从而得到第一注意力图和第二注意力图。其中，映射方式可以采用线性映射、高斯映射等多种映射方式。
在一些实施例中，第一计算设备120可以通过执行步骤420和430，基于第一注意力图、第二注意力图和分割图像块的位置信息，生成第一注意图和第二注意力图。
步骤420，基于每个分割图像块的位置信息和病灶定位概率生成第一注意力图。
在一些实施例中，在得到每个分割图像块的病灶定位概率后，第一计算设备120可以根据每个分割图像块的位置信息将所有病灶定位概率进行合并，从而得到第一注意力图。
在一些实施例中，第一计算设备120可以获取每个分割图像块所对应的权重，基于每个分割图像块的权重和病灶定位概率来生成第一注意力图。其中，权重可以由数字、表格、数据库、权重图(例如，高斯概率图)等各种形式来表示。在一些实施例中，权重图可以包括该图像块中元素的权重值，权重值可以反映该元素的模型输出结果(即病灶定位概率)的置信度。通常情况下，模型对图像中心区域的输出结果更加准确，而边缘区域的输出结果准确性较差，因此，高斯概率图中，通常中间区域的权重值高，边缘区域的权重值低。在一些实施例中，对于不同分割图像块，第一计算设备120可以使得权重均为中间区域 值较大，边缘区域值较小。在一些实施例中，不同分割图像块对应的权重可以是相同的。
在一些实施例中，对每个分割图像块，第一计算设备120可以基于其对应的权重更新该分割图像块的病灶定位概率，得到更新后的病灶定位概率。具体来说，第一计算设备120可以将该图像块的权重和病灶定位概率相乘，即将该图像块中元素的权重与其对应的病灶定位概率相乘，得到该元素更新后的病灶定位概率。图像块的更新后的病灶定位概率可以包括图像块中元素的更新后的病灶定位概率。
在一些实施例中，第一计算设备120可以根据每个分割图像块在初始医学图像中的位置信息，将每个图像块更新后的病灶定位概率合并，得到该初始医学图像对应的第一注意力图。在一些实施例中，在合并时，对初始医学图像中的每个元素，第一计算设备120可以根据图像块的位置信息确定该元素所在的对应图像块，以及该元素在对应图像块的更新后的病灶定位概率中的对应值。确定每个元素的对应值后，可以将所有的对应值进行加和求平均，从而得到该元素的第一注意力值。每个元素的第一注意力值确定后，即可生成第一注意力图。在一些实施例中，图像块的权重和在初始医学图像中的位置可以存储在对照表中。
在一些实施例中，病灶定位概率、更新后的病灶定位概率、第一注意力图等都可以以伪色彩图像的形式来呈现，其中的元素值可以用RGB值来反映。例如，病灶定位概率用原始伪色彩图像呈现。第一计算设备120可以根据病灶定位概率和RGB值的对应关系将每个图像块的病灶定位概率转化为原始伪色彩图像。第一计算设备120可以将图像块的原始伪色彩图像和权重相乘后，得到更新后的伪色彩图像(即更新后的病灶定位概率)。合并时，初始医学图像中的每个元素，第一计算设备120可以在更新后的伪色彩图像中找到该元素对应的RGB值，对该元素的RGB值进行求和平均，得到该元素最终的RGB值。每个元素的最终的RGB值可以组成第一注意力图对应的伪色彩图像，其中，每个元素的RGB值可以与其第一注意力值一一对应。在一些实施例中，病灶定位概率与RGB值的对应关系可以用信息对应表来存储。
在一些实施例中，第一计算设备120可以直接根据每个图像块的位置信息和病灶定位概率生成第一注意力图。具体来说，第一计算设备120可以直接根据每个分割图像块在初始医学图像中的位置信息，将每个图像块的病灶定位概率合并，得到该初始医学图像对应的第一注意力图。
在一些实施例中，对初始医学图像中的每个元素，第一计算设备120可以根据图像块的位置确定该元素所在的图像块、该元素在图像块中的病灶定位概率，对该元素的所有概率进行加和平均，得到该元素对应的第一注意力值。
步骤430，基于每个分割图像块的位置信息和病灶分类概率生成第二注意力图。
在一些实施例中，在得到每个分割图像块的病灶分类概率后，第一计算设备120可以根据每个分割图像块的位置信息、对应的权重图和病灶分类概率，确定初始医学图像中每个元素的第二注意力值，从而得到第二注意力图。
在一些实施例中，第一计算设备120可以直接根据每个分割图像块的位置信息和病灶分类概率生成第二注意力图。
基于每个分割图像块的位置信息和病灶分类概率生成第二注意力图的具体方法与生成第一注意力图的方法类似，区别在于第一注意力图基于病灶定位概率生成，详细内容可以参见步骤420的相关描述，在此不再赘述。
在一些实施例中，第一计算设备120可以将各分割图像块及其位置信息输入第一模型，得到输出的第一注意力图和第二注意力图。即将基于每个分割图像块的位置信息、病灶定位概率和病灶分类概率生成第一注意力图和第二注意力图的步骤集成在第一模型中。
在一些实施例中，第一计算设备120可以将初始医学图像输入第一模型，得到输出的第一注意力图和第二注意力图。即在上一个实施例基础上，进一步将图像分块的步骤也集成到第一模型中。
在一些实施例中，当存在多个病灶定位概率(例如，对应乳腺癌概率的概率、对应乳腺肿块概率的概率等)和多个病灶分类概率(例如，对应良恶性分类概率的概率、对应钙化分类概率的概率等)时，第一计算设备120可以综合这些概率生成第一注意力图和/或第二注意力图。
在一些实施例中，第一计算设备120可以根据最终要实现的目标来选择特定的病灶定位概率和病灶分类概率，用于生成第一注意力图和/或第二注意力图。例如，最终目标为进行乳腺癌识别，而乳腺癌和乳腺腺体密度相关度高，则可以选择对应乳腺癌概率的病灶定位概率和对应乳腺密度分类概率的病灶分类 概率。
在一些实施例中，第一计算设备120可以对每个病灶定位概率生成对应的初始第一注意力图，根据每个病灶定位概率和最终要实现的目标的关联性，设置每个初始第一注意力图的权重，对多个初始第一注意力图进行加权求和，生成最终的基于初始医学图像的第一注意力图。在一些实施例中，第一计算设备120可以对每个病灶分类概率生成对应的初始第二注意力图，根据每个病灶分类概率和最终要实现的目标的关联性，设置每个初始第二注意力图的权重，对多个初始第二注意力图进行加权求和，生成最终的基于初始医学图像的第二注意力图。例如，已得到的病灶分类概率中包括乳腺密度分类概率和乳腺扭曲分类概率，最终目标为进行乳腺癌诊断。而乳腺癌诊断与乳腺密度的判断结果关联度较高，则可以将由乳腺密度分类概率得到的初始第二注意力图的权重设置为较大值，将由乳腺扭曲分类概率得到的初始第二注意力图的权重设置为较小值；基于这两个初始第二注意力图的权重，对这两个初始第二注意力图进行加权求和处理，得到最终的基于初始医学图像的第二注意力图。
本说明书一些实施例中，通过基于分割图像块得到小块级别的病灶定位概率和病灶分类概率，可以得到比传统方式更为精细的病灶定位结果和病灶分类结果，提高了病灶分析的精确性；通过将分割图像块的病灶定位概率和病灶分类概率进行合并，从而得到了对初始医学图像的病灶定位概率和病灶分类概率的信息进行标注的图像，使得后续的图像检测能够更加具有针对性(例如，只对病灶存在概率高于阈值的区域进行检测、只对特定类型的病灶区域进行检测)，提高了图像检测的效率和准确性。
图5是根据本说明书一些实施例所示的图像检测方法的示例性流程图。
如图5所示，流程500包括下述步骤。在一些实施例中，流程500可以由第一计算设备120或检测结果生成模块230执行。
在一些实施例中，通过执行流程500所示的步骤，第一计算设备120可以基于至少一张注意力图(例如，第一注意力图、第二注意力图)和初始医学图像，使用第二模型生成目标对象的检测结果，其中，第二模型为训练好的机器学习模型。
步骤510，对多张初始医学图像中的每一张，利用第二模型处理第一注意力图和第二注意力图中的至少一个，以及初始医学图像，得到初始医学图像的初始检测结果。
在一些实施例中，多张初始医学图像可以包括对应多个视角的多张初始医学图像、多个不同的时间采集的相同视角的多张初始医学图像、多张不同模态的初始医学图像等。在一些实施例中，这些初始医学图像中的每一张都可以对应至少一张注意力图，其中，这些注意力图可以是基于这些初始医学图像的分割图像块生成的，例如，第一注意力图和/或第二注意力图。在一些实施例中，对多张初始医学图像中的每一张，第一计算设备120可以将初始医学图像，以及其对应的至少一张注意力图(例如，第一注意力图和/或第二注意力图)输入第二模型，得到输出的该初始医学图像对应的检测结果。
在一些实施例中，初始医学图像对应的检测结果可以包括病灶定位结果、病灶分类结果中的至少一种。其中，病灶定位结果包括病灶(例如，乳腺癌症病灶、乳腺钙化病灶、乳腺血管钙化病灶等)的位置、范围等，病灶的分类结果包括病灶的类型(例如，乳腺的良恶性、乳腺钙化、乳腺肿块性质等)、分级(例如，乳腺BI-RADS分级等)等。在一些实施例中，初始医学图像对应的检测结果可以包括初始医学图像的图像类型确定结果(例如，乳腺乳头凹陷类型的图像、乳腺腺体密度类型的图像等)。在一些实施例中，病灶定位结果和病灶分类结果可以通过概率形式表示。
在一些实施例中，第二模型可以为U-Net等卷积神经网络(Convolutional Neural Networks,CNN)。在一些实施例中，第二模型可以包括各种机器学习模型，例如，卷积神经网络、全卷积网络(Fully Convolutional Networks，FCN)等神经网络模型。
在一些实施例中，第二模型可以通过将医学图像样本和对应的注意力图样本作为训练样本对初始第二模型(即未经过训练的第二模型)进行无监督/有监督的训练获得，其中，这些医学图像样本可以包括各种类型的医学图像，例如，有病灶、无病灶、多个视角等。在一些实施例中，第二模型可以通过单一算法、集成算法、医学图像样本和对应的注意力图样本对初始第二模型进行训练获得。在一些实施例中，第二模型可以通过与第一模型联合训练获得，其中，第一模型的输出可以作为第二模型的训练样本。关于如何训练得到第二模型的更多内容，可以参见步骤740的相关描述，在此不再赘述。
在一些实施例中，第一计算设备120可以通过步骤330中描述的其他方式来确定初始医学图像的初始检测结果，例如，通过人工处理等方式。
步骤520，基于多张初始医学图像的初始检测结果，生成目标对象的检测结果。
在一些实施例中，在得到多个初始医学图像的初始检测结果之后，第一计算设备120可以综合这些初始医学图像的初始检测结果，生成目标对象的检测结果。例如，多个初始医学图像可以包括两个视角的医学图像，其中，第一视角可以为头尾相视角(craniocaudal(CC)view)，第二视角可以为内外斜位相视角(mediolateral oblique(MLO)view，即拍摄时人倾斜着侧躺)，第一计算设备120可以分别用两个视角的图像获得两个检测结果，再将这两个检测结果平均，将平均后得到的结果作为目标对象的检测结果。
在一些实施例中，对于以概率形式表示的病灶定位结果和病灶分类结果，第一计算设备120可以将各个初始医学图像的初始检测结果进行加权平均，得到加权后的概率值并将其作为目标对象的检测结果。其中，权重值可以基于经验、历史数据等确定。
在一些实施例中，第一计算设备120可以将多张初始医学图像及对应的注意力图输入第二模型，直接得到输出的目标对象的检测结果。即将综合多张初始医学图像的初始检测结果的步骤集成到第二模型中。
本说明书一些实施例中，通过基于多个视角多种医学图像和对应的注意力图得到目标对象的检测结果，提高了诊断准确率；通过使用机器学习模型提高了诊断效率，同时减轻了医生的负担。
在一些实施例中，可以只利用单张初始医学图像，生成目标对象的检测结果。例如，可以将该初始医学图像和对应的至少一张注意力图输入第二模型，得到所述检测结果。
图6是根据本说明书一些实施例所示的图像检测方法的示意图。
如图6所示，第一计算设备120可以将初始医学图像610进行分块，从而得到分割图像块620，其中，分割图像块620可以包括多个。关于如何对初始医学图像进行分块的更多内容，可以参见步骤310的相关描述，在此不再赘述。
在得到分割图像块620之后，第一计算设备120可以将每个分割图像块620分别输入第一模型630，得到输出的病灶定位概率631和病灶分类概率632。第一计算设备120可以根据每个分割图像块620的位置信息640和病灶定位概率631得到第一注意力图650，和/或根据每个分割图像块620的位置信息640和病灶分类概率632得到第二注意力图660。其中，位置信息640为分割图像块620在初始医学图像610中的位置信息。关于如何得到第一注意力图和第二注意力图的更多内容，可以参见步骤320及图4的相关描述，在此不再赘述。
在得到第一注意力图650和第二注意力图660之后，第一计算设备120可以将初始医学图像610、第一注意力图650和/或第二注意力图660输入第二模型670，得到检测结果680。其中，检测结果680可以为对应初始医学图像610的检测结果或目标对象的检测结果。
图7是根据本说明书一些实施例所示的模型训练方法的示例性流程图。
如图7所示，流程700包括下述步骤。在一些实施例中，流程700可以由第二计算设备130或模型训练模块执行。在一些实施例中，通过执行流程700所示的步骤，第二计算设备130可以对初始第一模型和初始第二模型进行共同训练，得到训练好的第一模型和第二模型。
步骤710，获取医学图像样本的分割图像块样本。
在一些实施例中，第二计算设备130可以通过各种方式获取医学图像作为医学图像样本，然后将这些医学图像样本进行分块操作，得到医学图像样本的分割图像块样本。在一些实施例中，第二计算设备130可以指示第一计算设备120将医学图像样本进行分块。
在一些实施例中，分割图像块样本可以在标注有病灶区域的医学图像样本中采样获得。在一些实施例中，在创建包含病灶的图像块时，第二计算设备130可以选择病灶边界内的随机位置作为每一个图像块的中心。如果得到的图像块包含标注的病灶像素数量不足或者该图像块包含的感兴趣区域(例如，乳腺等)的范围过小，则可以重新采样一个新的图像块，然后采用数据增强的方式(例如，随机旋转、随机翻转、随机缩放等)，来增强样本数据集。
关于如何获取医学图像以及对图像进行分块操作的更多内容，可以参见步骤310的相关描述，在 此不再赘述。
步骤720，将分割图像块样本作为第一模型的训练样本，训练初始第一模型，得到第一模型。
在一些实施例中，第二计算设备130可以将分割图像块样本作为第一模型的训练样本，将分割图像块样本对应的病灶分类概率样本和病灶定位概率样本作为训练标签(通过人工标注或历史数据等获得)，训练初始第一模型，从而得到第一模型。
在一些实施例中，第二计算设备130可以使用分割图像块样本作为初始第一模型的输入，将初始第一模型输出的预测病灶分类概率和预测病灶定位概率与训练标签相比较。根据比较结果可以确定损失函数的值。例如，根据预测病灶定位概率和病灶定位概率样本的比较结果可以得到分割损失(例如，Jaccard loss和focal loss)的第一值，根据预测病灶分类概率和病灶分类概率样本可以得到分类损失(例如，cross entropy loss)的第二值。损失函数值的值为第一值和第二值的加权和。根据损失函数的值迭代更新第一初始模型的参数值，直到预设条件被满足。其中，预设条件可以包括达到预设迭代次数、预测病灶分类概率和预测病灶定位概率与训练标签的差异小于阈值等。
步骤730，基于医学图像样本和第一模型，生成医学图像样本对应的注意力图样本。
在一些实施例中，在得到训练好的第一模型后，第二计算设备130可以根据医学图像样本和第一模型，生成医学图像样本对应的注意力图样本。具体来说，第二计算设备130可以将每个分割图像块样本分别输入第一模型，得到输出的病灶分类概率和病灶定位概率。然后基于每个分割图像块样本的位置信息和病灶分类概率和/或病灶定位概率，第二计算设备130可以生成医学图像样本对应的第一注意力图样本和/或第二注意力图样本。关于如何生成注意力图的更多内容，可以参见步骤320和图4的相关描述，在此不再赘述。
步骤740，将注意力图样本和医学图像样本作为第二模型的训练样本，训练初始第二模型，得到第二模型。
在一些实施例中，在得到注意力图样本之后，第二计算设备130可以将医学图像样本和与其对应的注意力图样本作为第二模型的训练样本，将与医学图像样本对应的检测结果样本作为训练标签(通过人工标注或历史数据等获得)，训练初始第二模型，从而得到第二模型。其中，用于训练初始第二模型的损失函数的类型可以根据第二模型输出的检测结果类型确定。以检测结果为乳腺肿瘤良恶性分类为例，损失函数可以为交叉熵(例如，二分类的交叉熵(binary cross entropy loss))。训练第二模型的具体方式与训练第一模型类似，在此不再赘述。
在一些实施例中，第一模型和第二模型也可以各自进行单独训练，其中，这两个模型用于训练的各自的医学图像样本可以相同，也可以不相同。
在一些实施例中，第一模型和第二模型也可以进行联合训练。其中，联合训练的训练样本可以包括医学图像样本和对应的检测结果样本。在训练过程中，第二计算设备130可以将医学图像样本的分割图像块样本输入初始第一模型，得到输出的预测病灶分类概率和预测病灶定位概率；基于每个分割图像块样本的位置信息和预测病灶分类概率和/或预测病灶定位概率，生成医学图像样本对应的第一注意力图样本和/或第二注意力图样本；然后将医学图像样本和生成的第一注意力图样本和/或第二注意力图样本输入初始第二模型，基于初始第二模型输出的预测检测结果对初始第一模型和初始第二模型的网络参数同时进行迭代调整和设置，直到预设条件被满足。其中，预设条件可以包括达到预设迭代次数、预测检测结果与训练标签的差异小于阈值等。
应当注意的是，上述有关流程300、400、500、700的描述仅仅是为了示例和说明，而不限定本说明书的适用范围。对于本领域技术人员来说，在本说明书的指导下可以对流程300、400、500、700进行各种修正和改变。然而，这些修正和改变仍在本说明书的范围之内。例如，步骤420和430的顺序可以交换。
图8是根据本说明书一些实施例所示的第一模型的结构示意图。
如图8所示，第一模型800可以包括编码器810、解码器820和分类框架830。
第一模型800的输入为分割图像块850。在一些实施例中，编码器810可以用于提取分割图像块850的编码特征信息；解码器820可以用于基于编码特征信息提取分割图像块850的解码特征信息，并基于解码特征信息生成分割图像块850对应的病灶定位概率860；分类框架830可以用于基于编码特征信息 和解码特征信息中的至少一部分生成分割图像块850对应的病灶分类概率870。
在一些实施例中，编码器810可以通过卷积操作、归一化、降采样等方式从分割图像块850中提取编码特征信息。例如，如图8所示，编码器810中的编码层可以包括编码层811-1、811-2、811-3、…、811-n，其中，n≥2。在一些实施例中，编码器810可以通过编码层进行特征提取，其中，每两个编码层之间包括一次降采样操作。例如，如果n＝5，则编码器810可以对分割图像块850进行4次降采样操作。
在一些实施例中，解码器820可以通过卷积操作、升采样等方式从编码特征信息中提取分割图像块850的解码特征信息。在一些实施例中，升采样的次数可以与降采样的次数相同。例如，如图8所示，与编码层对应，解码器820中的解码层可以包括解码层821-1、821-2、821-3、…、821-n。在一些实施例中，解码器820可以通过解码层进行特征解码，其中，每两个解码层之间包括一次升采样操作。例如，如果n＝5，则解码器820可以进行4次升采样操作。在一些实施例中，编码器810可以将每一次特征提取后得到的编码特征信息输入解码层中。例如，在编码层811-1得到的编码特征信息可以输入解码层821-2中。
在一些实施例中，第一模型800的解码器820可以包括注意力块，如图8所示，解码器820中的注意力块可以与其解码层一一对应，包括注意力块822-1、822-2、822-3、…、822-n。其中，注意力块可以用于基于空间特征的权重、通道特征的权重和原始特征，对空间特征和通道特征进行加强处理。例如，注意力块822-2可以包含在解码层821-2之中，同时对空间特征和通道特征计算权重，然后和原始特征进行相乘和相加得到新的特征值，即新的特征值＝空间特征权重*原始特征+通道特征权重*原始特征。
在一些实施例中，第一模型800还可以包括归一化层，归一化层可以用于连接编码器和解码器，并对编码特征信息进行归一化处理。如图8所示，第一模型800的归一化层包括归一化层840-1、840-2、840-3、…、840-n。归一化层可以位于编码器810和解码器820之间，每一归一化层连接对应的编码层和解码层。
在一些实施例中，编码器810可以将每一次特征提取后得到的编码特征信息输入对应的归一化层；该归一化层对编码特征信息进行归一化处理，将归一化处理后的编码特征信息输入对应的解码层。例如，编码层811-2将特征提取得到的编码特征信息输入归一化层840-2中，归一化层840-2对该编码特征信息进行归一化处理并将处理后信息输入解码层821-2中。在一些实施例中，归一化操作可以包括对不同尺度的编码特征信息(例如，encoder feature map)进行归一化。
在一些实施例中，解码器820可以将多次升采样后得到的分割图像块850最终的编码特征信息进行融合，从而得到分割图像块850对应的病灶定位概率860。
在一些实施例中，分类框架830可以将至少一个编码层输出的编码特征信息和至少一个解码层输出的解码特征信息融合，得到分割图像块850对应的病灶分类概率870。例如，如果图8中的n＝5，则分类框架830可以将编码层811-5输出编码特征信息、编码层811-4输出的解码特征信息和解码层821-5输出的解码特征信息进行融合，得到病灶分类概率870。
本说明书实施例可能带来的有益效果包括但不限于：(1)通过将医学图像(例如，乳腺钼靶图像等)分割为图像块，将其输入机器学习模型得到病灶定位概率和病灶分类概率，并进一步通过机器学习模型得到疾病的诊断结果，从而可以精准地实现对病灶(例如，乳腺病灶等)的定位，避免了假阳性的检测结果；而且，能够结合精准定位的病灶和医学图像，给出合理的疾病辅助诊断结果，为医生提供客观标准，减少了主观因素影响，从而提高了诊断准确性；通过使用机器学习模型在一定程度上减少了医生工作量，提高了诊断效率，也避免了病人不必要的额外检查；(2)通过使用多任务的机器学习模型，可以同时输出病灶定位概率和病灶分类概率，从而提高了模型使用的便利性，同时能够基于多种检测结果得到准确的诊断结果；(3)通过机器学习模型得到的检测结果(病灶定位概率和病灶分类概率)以及诊断结果可以包括多种类型，从而提高了模型的适应性，很好地满足了多样化的诊断需求。需要说明的是，不同实施例可能产生的有益效果不同，在不同的实施例里，可能产生的有益效果可以是以上任意一种或几种的组合，也可以是其他任何可能获得的有益效果。
上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述详细披露仅仅作为示例，而并不构成对本说明书的限定。虽然此处并没有明确说明，本领域技术人员可能会对本说明书进行各种修改、改进和修正。该类修改、改进和修正在本说明书中被建议，所以该类修改、改进、修正仍属于本说明书示 范实施例的精神和范围。
同时，本说明书使用了特定词语来描述本说明书的实施例。如“一个实施例”、“一实施例”、和/或“一些实施例”意指与本说明书至少一个实施例相关的某一特征、结构或特点。因此，应强调并注意的是，本说明书中在不同位置两次或多次提及的“一实施例”或“一个实施例”或“一个替代性实施例”并不一定是指同一实施例。此外，本说明书的一个或多个实施例中的某些特征、结构或特点可以进行适当的组合。
此外，除非权利要求中明确说明，本说明书所述处理元素和序列的顺序、数字字母的使用、或其他名称的使用，并非用于限定本说明书流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本说明书实施例实质和范围的修正和等价组合。例如，虽然以上所描述的系统组件可以通过硬件设备实现，但是也可以只通过软件的解决方案得以实现，如在现有的服务器或移动设备上安装所描述的系统。
同理，应当注意的是，为了简化本说明书披露的表述，从而帮助对一个或多个发明实施例的理解，前文对本说明书实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本说明书对象所需要的特征比权利要求中提及的特征多。实际上，实施例的特征要少于上述披露的单个实施例的全部特征。
一些实施例中使用了描述成分、属性数量的数字，应当理解的是，此类用于实施例描述的数字，在一些示例中使用了修饰词“大约”、“近似”或“大体上”来修饰。除非另外说明，“大约”、“近似”或“大体上”表明所述数字允许有±20％的变化。相应地，在一些实施例中，说明书和权利要求中使用的数值参数均为近似值，该近似值根据个别实施例所需特点可以发生改变。在一些实施例中，数值参数应考虑规定的有效数位并采用一般位数保留的方法。尽管本说明书一些实施例中用于确认其范围广度的数值域和参数为近似值，在具体实施例中，此类数值的设定在可行范围内尽可能精确。
针对本说明书引用的每个专利、专利申请、专利申请公开物和其他材料，如文章、书籍、说明书、出版物、文档等，特此将其全部内容并入本说明书作为参考。与本说明书内容不一致或产生冲突的申请历史文件除外，对本说明书权利要求最广范围有限制的文件(当前或之后附加于本说明书中的)也除外。需要说明的是，如果本说明书附属材料中的描述、定义、和/或术语的使用与本说明书所述内容有不一致或冲突的地方，以本说明书的描述、定义和/或术语的使用为准。
最后，应当理解的是，本说明书中所述实施例仅用以说明本说明书实施例的原则。其他的变形也可能属于本说明书的范围。因此，作为示例而非限制，本说明书实施例的替代配置可视为与本说明书的教导一致。相应地，本说明书的实施例不仅限于本说明书明确介绍和描述的实施例。

Claims (28)

  1. 一种图像检测方法，包括：
    获取目标对象的初始医学图像中的多个分割图像块；
    基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，所述至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，所述第一注意力图与病灶定位相关，所述第二注意力图与病灶分类相关；以及
    基于所述至少一张注意力图，生成所述目标对象的检测结果。
  2. 如权利要求1所述的方法，所述第一注意力图包括所述初始医学图像中的元素的第一注意力值，所述第一注意力值与所述元素属于病灶的概率相关，所述第二注意力图包括所述初始医学图像中的元素的第二注意力值，所述第二注意力值与所述元素的病灶分类相关。
  3. 如权利要求1所述的方法，所述基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，包括：
    对所述多个分割图像块中的每个分割图像块，使用第一模型生成所述分割图像块的病灶定位概率和病灶分类概率，所述第一模型为训练好的机器学习模型；以及
    基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图。
  4. 如权利要求3所述的方法，所述基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图，包括：
    基于所述每个分割图像块的位置信息和所述病灶定位概率生成所述第一注意力图；以及
    基于所述每个分割图像块的所述位置信息和所述病灶分类概率生成所述第二注意力图。
  5. 如权利要求3所述的方法，所述基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图，包括：
    将所述病灶定位概率和所述病灶分类概率重新映射到所述初始医学图像，得到所述第一注意力图和所述第二注意力图，所述映射包括线性映射和高斯映射中的至少一种。
  6. 如权利要求3所述的方法，所述基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，进一步包括：
    获取所述多个分割图像块中的每个分割图像块对应的权重，其中所述第一注意力图和所述第二注意力图进一步基于所述权重生成。
  7. 如权利要求3所述的方法，所述多个分割图像块至少包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块，所述至少一个第一分割图像块用于生成所述病灶定位概率，所述至少一个第二分割图像块用于生成所述病灶分类概率。
  8. 如权利要求3所述的方法，
    所述第一模型包括编码器、解码器和分类框架，
    对所述每个分割图像块，
    所述编码器用于提取所述分割图像块的编码特征信息，
    所述解码器用于基于所述编码特征信息提取所述分割图像块的解码特征信息，并基于所述解码特征信息生成所述分割图像块的所述病灶定位概率，
    所述分类框架用于基于所述编码特征信息和所述解码特征信息中的至少一部分生成所述分割图像块的所述病灶分类概率。
  9. 如权利要求8所述的方法，所述第一模型还包括归一化层，所述归一化层用于连接所述编码器和所 述解码器，并对所述编码特征信息进行归一化处理。
  10. 如权利要求8所述的方法，所述解码器包括注意力块，所述注意力块用于基于空间特征的权重、通道特征的权重和原始特征，对所述空间特征和所述通道特征进行加强处理。
  11. 如权利要求1所述的方法，所述基于所述至少一张注意力图，生成所述目标对象的检测结果，包括：
    基于所述至少一张注意力图和所述初始医学图像，使用第二模型生成所述目标对象的检测结果，所述第二模型为训练好的机器学习模型。
  12. 如权利要求11所述的方法，所述初始医学图像包括对应多个视角的多张初始医学图像，所述至少一张注意力图包括每张所述初始医学图像对应的至少一张注意力图，所述基于所述至少一张注意力图和所述初始医学图像，使用第二模型生成所述目标对象的检测结果包括：
    对所述多张初始医学图像中的每一张，利用所述第二模型处理所述初始医学图像及其对应的至少一张注意力图，得到所述初始医学图像的初始检测结果；以及
    基于所述多张初始医学图像的初始检测结果，生成所述目标对象的检测结果。
  13. 如权利要求1所述的方法，所述至少一张注意力图通过第一模型获得，所述目标对象的检测结果通过第二模型获得，所述第一模型和所述第二模型通过以下方式训练获得：
    获取医学图像样本的分割图像块样本；
    将所述分割图像块样本作为所述第一模型的训练样本，训练初始第一模型，得到所述第一模型；
    基于所述医学图像样本和所述第一模型，生成所述医学图像样本对应的注意力图样本；
    将所述注意力图样本和所述医学图像样本作为所述第二模型的训练样本，训练初始第二模型，得到所述第二模型。
  14. 一种图像检测系统，包括图像块获取模块、注意力图生成模块和检测结果生成模块；
    所述图像块获取模块用于获取目标对象的初始医学图像中的多个分割图像块；
    所述注意力图生成模块用于基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，所述至少一张注意力图包括第一注意力图和第二注意力图中的至少一个，所述第一注意力图与病灶定位相关，所述第二注意力图与病灶分类相关；以及
    所述检测结果生成模块用于基于所述至少一张注意力图，生成所述目标对象的检测结果。
  15. 如权利要求14所述的系统，所述第一注意力图包括所述初始医学图像中的元素的第一注意力值，所述第一注意力值与所述元素属于病灶的概率相关，所述第二注意力图包括所述初始医学图像中的元素的第二注意力值，所述第二注意力值与所述元素的病灶分类相关。
  16. 如权利要求14所述的系统，所述基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，包括：
    对所述多个分割图像块中的每个分割图像块，使用第一模型生成所述分割图像块的病灶定位概率和病灶分类概率，所述第一模型为训练好的机器学习模型；以及
    基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图。
  17. 如权利要求16所述的系统，所述基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图，包括：
    基于所述每个分割图像块的位置信息和所述病灶定位概率生成所述第一注意力图；以及
    基于所述每个分割图像块的所述位置信息和所述病灶分类概率生成所述第二注意力图。
  18. 如权利要求16所述的系统，所述基于所述病灶定位概率和所述病灶分类概率得到所述第一注意力图和所述第二注意力图，包括：
    将所述病灶定位概率和所述病灶分类概率重新映射到所述初始医学图像，得到所述第一注意力图和所述第二注意力图，所述映射包括线性映射和高斯映射中的至少一种。
  19. 如权利要求16所述的系统，所述基于所述多个分割图像块，生成所述初始医学图像的至少一张注意力图，进一步包括：
    获取所述多个分割图像块中的每个分割图像块对应的权重，其中所述第一注意力图和所述第二注意力图进一步基于所述权重生成。
  20. 如权利要求16所述的系统，所述多个分割图像块至少包括第一大小的至少一个第一分割图像块和第二大小的至少一个第二分割图像块，所述至少一个第一分割图像块用于生成所述病灶定位概率，所述至少一个第二分割图像块用于生成所述病灶分类概率。
  21. 如权利要求16所述的系统，
    所述第一模型包括编码器、解码器和分类框架，
    对所述每个分割图像块，
    所述编码器用于提取所述分割图像块的编码特征信息，
    所述解码器用于基于所述编码特征信息提取所述分割图像块的解码特征信息，并基于所述解码特征信息生成所述分割图像块的所述病灶定位概率，
    所述分类框架用于基于所述编码特征信息和所述解码特征信息中的至少一部分生成所述分割图像块的所述病灶分类概率。
  22. 如权利要求21所述的系统，所述第一模型还包括归一化层，所述归一化层用于连接所述编码器和所述解码器，并对所述编码特征信息进行归一化处理。
  23. 如权利要求21所述的系统，所述解码器包括注意力块，所述注意力块用于基于空间特征的权重、通道特征的权重和原始特征，对所述空间特征和所述通道特征进行加强处理。
  24. 如权利要求14所述的系统，所述基于所述至少一张注意力图，生成所述目标对象的检测结果，包括：
    基于所述至少一张注意力图和所述初始医学图像，使用第二模型生成所述目标对象的检测结果，所述第二模型为训练好的机器学习模型。
  25. 如权利要求24所述的系统，所述初始医学图像包括对应多个视角的多张初始医学图像，所述至少一张注意力图包括每张所述初始医学图像对应的至少一张注意力图，所述基于所述至少一张注意力图和所述初始医学图像，使用第二模型生成所述目标对象的检测结果包括：
    对所述多张初始医学图像中的每一张，利用所述第二模型处理所述初始医学图像及其对应的至少一张注意力图，得到所述初始医学图像的初始检测结果；以及
    基于所述多张初始医学图像的初始检测结果，生成所述目标对象的检测结果。
  26. 如权利要求14所述的系统，所述至少一张注意力图通过第一模型获得，所述目标对象的检测结果通过第二模型获得，所述第一模型和所述第二模型通过以下方式训练获得：
    获取医学图像样本的分割图像块样本；
    将所述分割图像块样本作为所述第一模型的训练样本，训练初始第一模型，得到所述第一模型；
    基于所述医学图像样本和所述第一模型，生成所述医学图像样本对应的注意力图样本；
    将所述注意力图样本和所述医学图像样本作为所述第二模型的训练样本，训练初始第二模型，得到所述第二模型。
  27. 一种图像检测装置，包括处理器，所述处理器用于执行权利要求1～13中任一项所述的方法。
  28. 一种计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行如权利要求1～13中任一项所述的方法。
PCT/CN2023/088226 2022-04-14 2023-04-13 图像检测方法、系统、装置及存储介质 WO2023198166A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210389475.9A CN114820483A (zh) 2022-04-14 2022-04-14 图像检测方法、装置及计算机设备
CN202210389475.9 2022-04-14

Publications (1)

Publication Number Publication Date
WO2023198166A1 true WO2023198166A1 (zh) 2023-10-19

Family

ID=82536561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/088226 WO2023198166A1 (zh) 2022-04-14 2023-04-13 图像检测方法、系统、装置及存储介质

Country Status (2)

Country Link
CN (1) CN114820483A (zh)
WO (1) WO2023198166A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114820483A (zh) * 2022-04-14 2022-07-29 北京联影智能影像技术研究院 图像检测方法、装置及计算机设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109447966A (zh) * 2018-10-26 2019-03-08 科大讯飞股份有限公司 医学图像的病灶定位识别方法、装置、设备及存储介质
CN110706793A (zh) * 2019-09-25 2020-01-17 天津大学 一种基于注意力机制的甲状腺结节半监督分割方法
CN110807788A (zh) * 2019-10-21 2020-02-18 腾讯科技（深圳）有限公司 医学图像处理方法、装置、电子设备及计算机存储介质
CN113012166A (zh) * 2021-03-19 2021-06-22 北京安德医智科技有限公司 颅内动脉瘤分割方法及装置、电子设备和存储介质
US20220109517A1 (en) * 2019-02-13 2022-04-07 Nippon Telegraph And Telephone Corporation Detection device, detection method, and program
CN114820483A (zh) * 2022-04-14 2022-07-29 北京联影智能影像技术研究院 图像检测方法、装置及计算机设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109447966A (zh) * 2018-10-26 2019-03-08 科大讯飞股份有限公司 医学图像的病灶定位识别方法、装置、设备及存储介质
US20220109517A1 (en) * 2019-02-13 2022-04-07 Nippon Telegraph And Telephone Corporation Detection device, detection method, and program
CN110706793A (zh) * 2019-09-25 2020-01-17 天津大学 一种基于注意力机制的甲状腺结节半监督分割方法
CN110807788A (zh) * 2019-10-21 2020-02-18 腾讯科技（深圳）有限公司 医学图像处理方法、装置、电子设备及计算机存储介质
CN113012166A (zh) * 2021-03-19 2021-06-22 北京安德医智科技有限公司 颅内动脉瘤分割方法及装置、电子设备和存储介质
CN114820483A (zh) * 2022-04-14 2022-07-29 北京联影智能影像技术研究院 图像检测方法、装置及计算机设备

Also Published As

Publication number Publication date
CN114820483A (zh) 2022-07-29

Similar Documents

Publication Publication Date Title
US8238637B2 (en) Computer-aided diagnosis of malignancies of suspect regions and false positives in images
WO2019062846A1 (zh) 结合影像识别与报告编辑的医学影像辅助诊断方法及系统
WO2021179491A1 (zh) 图像处理方法、装置、计算机设备和存储介质
WO2014113786A1 (en) Quantitative predictors of tumor severity
CN110796636A (zh) 基于卷积神经网络的ct图像骨质状况检测方法及装置
Heydarheydari et al. Auto-segmentation of head and neck tumors in positron emission tomography images using non-local means and morphological frameworks
Chae et al. Automatic lung segmentation for large-scale medical image management
CN110348477B (zh) 医疗图像处理方法、存储介质及计算机设备
CN114445334B (zh) 图像分析的方法、装置、设备及存储介质
WO2023198166A1 (zh) 图像检测方法、系统、装置及存储介质
CN112071418B (zh) 基于增强ct影像组学的胃癌腹膜转移的预测系统及方法
JP2022164527A (ja) 医用画像による腫瘍検出及び診断装置
Zhou et al. Interpreting medical images
Selim et al. CT image harmonization for enhancing radiomics studies
Gunasekaran Leveraging object detection for the identification of lung cancer
Peña-Solórzano et al. Findings from machine learning in clinical medical imaging applications–Lessons for translation to the forensic setting
Irene et al. Segmentation and approximation of blood volume in intracranial hemorrhage patients based on computed tomography scan images using deep learning method
Kaliyugarasan et al. Pulmonary nodule classification in lung cancer from 3D thoracic CT scans using fastai and MONAI
Shao et al. Application of an improved u2-net model in ultrasound median neural image segmentation
Zhou et al. Deep CTS: a deep neural network for identification MRI of carpal tunnel syndrome
CN111681205B (zh) 图像分析方法、计算机设备和存储介质
Jiang et al. A hybrid enhanced attention transformer network for medical ultrasound image segmentation
Zhang et al. Automatic parotid gland segmentation in MVCT using deep convolutional neural networks
Zhang et al. CCS-net: cascade detection network with the convolution kernel switch block and statistics optimal anchors block in hypopharyngeal cancer MRI
US20230115927A1 (en) Systems and methods for plaque identification, plaque composition analysis, and plaque stability detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787808

Country of ref document: EP

Kind code of ref document: A1