WO2022089236A1 - 基于人工智能的图像处理方法、装置、计算机设备和存储介质 - Google Patents

基于人工智能的图像处理方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2022089236A1
WO2022089236A1 PCT/CN2021/124384 CN2021124384W WO2022089236A1 WO 2022089236 A1 WO2022089236 A1 WO 2022089236A1 CN 2021124384 W CN2021124384 W CN 2021124384W WO 2022089236 A1 WO2022089236 A1 WO 2022089236A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target element
pixel
area
region
Prior art date
Application number
PCT/CN2021/124384
Other languages
English (en)
French (fr)
Inventor
王亮
姚建华
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP21884964.4A priority Critical patent/EP4198889A4/en
Priority to JP2023513664A priority patent/JP7412636B2/ja
Publication of WO2022089236A1 publication Critical patent/WO2022089236A1/zh
Priority to US17/960,804 priority patent/US20230023585A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present application relates to the field of computer technology, and in particular, to an artificial intelligence-based image processing method, apparatus, computer device, and storage medium.
  • Computer Vision is a simulation of biological vision using computers and related equipment. Its main task is to obtain three-dimensional information of the corresponding scene by processing the collected pictures or videos, just like humans and many other Like what creatures do every day.
  • object recognition and detection is a very basic and important research direction in computer vision. Object recognition and detection, that is, given an input picture, automatically find common objects in the picture, and output their category and location, such as face detection, vehicle detection, tissue detection, etc., in manufacturing, inspection , document analysis, medical diagnosis and military fields have been widely used.
  • an artificial intelligence-based image processing method According to various embodiments of the present application, an artificial intelligence-based image processing method, apparatus, computer device, and storage medium are provided.
  • An artificial intelligence-based image processing method executed by computer equipment comprising:
  • An artificial intelligence-based image processing device comprising:
  • Image acquisition module used to acquire the image to be processed
  • the element area detection module is used to detect the element area of the image to be processed to determine the element area in the image to be processed;
  • the target element area detection module is used to detect the target element area in the image to be processed by adopting the target element area detection method based on artificial intelligence;
  • an envelope generation module for finding an envelope for the detected target element region to generate the target element envelope region
  • the area fusion module is used to fuse the element area and the target element envelope area to obtain the outline of the target element area.
  • a computer device comprising a memory and one or more processors, the memory storing computer-readable instructions that, when executed by the processor, cause the one or more processors to perform the following: step:
  • One or more non-volatile readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:
  • Fig. 1 is the application environment diagram of the image processing method based on artificial intelligence in one embodiment
  • FIG. 2 is a schematic flowchart of an image processing method based on artificial intelligence in one embodiment
  • 3 is a schematic flowchart of element area detection in one embodiment
  • FIG. 4 is a schematic diagram of a tumor in an HE-stained image in one embodiment
  • FIG. 6 is a comparison diagram of an HE-stained WSI image and a PD-L1-stained WSI image in one embodiment
  • FIG. 7 is a comparison diagram of the HE-stained WSI image and the PD-L1-stained WSI image in another embodiment
  • FIG. 8 is a comparison diagram of an HE-stained WSI image and a PD-L1-stained WSI image in yet another embodiment
  • FIG. 9 is a schematic flowchart of an image processing method based on artificial intelligence in another embodiment.
  • FIG. 10 is a schematic diagram of the result of color segmentation processing in one embodiment
  • Fig. 11 is a schematic diagram of the result of binarization segmentation processing in the embodiment shown in Fig. 10;
  • Figure 12 is a stained image in one embodiment
  • Fig. 13 is a schematic diagram of the result of detecting the sliced tissue region in the embodiment shown in Fig. 12;
  • Figure 14 is a network model structure in one embodiment
  • 15 is a probability map of cancer cell regions output by a model in one embodiment
  • Fig. 16 is a schematic diagram of the result of the cancer cell region determined in the embodiment of Fig. 15;
  • Fig. 17 is a schematic diagram of a connected domain detection result in the embodiment of Fig. 16;
  • Figure 18 is a schematic diagram of the removal result after black hole removal in the embodiment of Figure 17;
  • Fig. 19 is a schematic diagram of the result after morphological expansion in the embodiment of Fig. 18;
  • Fig. 20 is a schematic diagram of the result of fusion in the embodiment shown in Fig. 16 and Fig. 18;
  • Fig. 21 is a schematic diagram of the result of superimposing dyed images in the embodiment of Fig. 20;
  • 22 is a structural block diagram of an image processing apparatus based on artificial intelligence in one embodiment
  • Figure 23 is a diagram of the internal structure of a computer device in one embodiment.
  • the artificial intelligence-based image processing method provided in this application is executed by a computer device, and can be specifically applied to the application environment shown in FIG. 1 .
  • the terminal 102 communicates with the server 104 through the network.
  • the terminal 102 collects the to-be-processed images that require image processing, such as images in various scenes such as landscape images, scene images, slice images, etc.
  • the terminal 102 sends the to-be-processed images to the server 104 for detection and division of the target element area contour.
  • the server 104 After receiving the to-be-processed image, the server 104 detects the target element area in the to-be-processed image through the target element area detection method based on artificial intelligence, searches for the envelope of the target element area to obtain the target element envelope area, and then fuses the target element area through the element area The determined element area and the target element envelope area are detected to obtain the outline of the target element area.
  • the server 104 may also perform image processing by directly acquiring the image to be processed from the database, so as to identify the contour of the target element region in the image to be processed.
  • the terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices with a shooting function, and can also be various camera devices or scanning devices and other devices capable of capturing images.
  • the server 104 It can be implemented with an independent server or a server cluster composed of multiple servers.
  • an image processing method based on artificial intelligence is provided, and the method is applied to the server in FIG. 1 as an example to illustrate, including the following steps:
  • Step 202 acquiring an image to be processed.
  • the image to be processed is an image that needs to be divided into regions, and the image to be processed can be an image taken or scanned in various scenes, such as a landscape image taken in a field scene, a scene image taken in a crowd gathering scene, or scanned in a medical scene. various types of images such as slice images.
  • the images to be processed include elements captured by shooting or scanning, such as trees captured in landscape images, people captured in scene images, etc., and the elements have different categories, such as trees of different species, people of different genders, etc.
  • the to-be-processed image can be captured by a photographing device, or scanned by a scanning device, and sent to the server through the network, and the server will perform identification and division processing on the received image to be processed by the contour of the target element area, so as to determine the to-be-processed image.
  • the distribution of the contour of the target element area in the image which is convenient for subsequent image analysis.
  • the target element can be set according to actual needs. For example, when the element included in the image to be processed is a tree, the target element can be a specific tree species set; for example, when the element included in the image to be processed is a cell, The target element can be a specific type of cell, such as leukocytes, somatic cells, cancer cells, etc.
  • Step 204 performing element area detection on the image to be processed to determine the element area in the image to be processed.
  • the element is the object captured or scanned in the image to be processed, and the area outline of the target element to be detected is a certain type of the element.
  • the target element can be different organs of the tree (such as leaves, tree trunks, etc.) or trees of different species types, etc.; for example, the element is a dog, and the target element can be different parts of the dog.
  • the element area refers to the distribution range of elements in the image to be processed, such as the distribution range of trees of various species in the image to be processed. In the specific application, the element is determined according to the actual scene of image processing.
  • the element can also be a cell in the sliced image, and the element area is the cell area in the sliced image, that is, the sliced tissue area.
  • Element area detection is used to detect the element area in the image to be processed.
  • various object detection algorithms can be used, such as R-CNN (Regions with Convolutional Neural Networks features, regions based on convolutional neural network features) algorithm, YOLO (You Only Look Once) algorithm, SSD (Single Shot Detector, single-shot detector) algorithm, etc., perform element area detection on the image to be processed to determine the element area in the image to be processed.
  • prior knowledge can also be used, such as prior knowledge of color features of elements and non-elements in the image to be processed, and color features of each pixel in the image to be processed can be used to detect element regions in the image to be processed.
  • the server determines the corresponding element area detection method according to the type characteristics of the elements in the image to be processed, and performs element area detection on the image to be processed through the element area detection method to determine the element area in the image to be processed. .
  • step 206 the target element area detection method based on artificial intelligence is used to detect the target element area in the image to be processed.
  • the target element may be a sub-category element in the element, that is, the target element belongs to the element that needs to be detected.
  • the target element area is the distribution range of the target element in the image to be processed.
  • the target element area detection method is implemented based on artificial intelligence, for example, the target element area detection can be realized based on a machine learning model.
  • FCN Full Convolutional Networks, fully convolutional network
  • SegNet A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, a deep convolutional encoding-decoding architecture for image segmentation
  • Linknet Relational Various image segmentation algorithm models such as Embedding for Scene Graph, the relational embedding of scene graph
  • the server when performing target element area detection on the image to be processed, the server detects the target element area based on artificial intelligence, such as using a pre-trained detection model to perform target element area detection on the image to be processed, and detects the target in the image to be processed. element area.
  • artificial intelligence such as using a pre-trained detection model to perform target element area detection on the image to be processed
  • Step 208 searching for an envelope for the detected target element area to generate the target element envelope area.
  • the target element envelope area is a preliminary target element area outline obtained by finding the envelope based on the target element area. Specifically, after the target element region detection method based on artificial intelligence detects the target element region in the image to be processed, the server searches for the envelope of the target element region.
  • the envelope of the element area, the target element envelope area is generated according to the area surrounded by the envelope, and by finding the envelope of the target element area, the influence of the error in the image processing process can be reduced, and the accurate target element envelope area can be obtained.
  • the server can connect the target element area to obtain the connected area, and then the server removes the non-target element area within the range of the connected area, so as to ensure that the target element recognition result is accurate within the range of the connected area, and finally
  • the server performs smooth processing on the connected areas with non-target element areas removed, such as smooth processing through morphological expansion to obtain the target element envelope area with smooth edges, and at the same time, the smaller areas are denoised through morphological expansion. , which improves the accuracy of the target element envelope region recognition and division.
  • Step 210 fuse the element area and the target element envelope area to obtain the outline of the target element area.
  • the two are fused to obtain the outline of the target element area, which overall shows the distribution range of the target element in the image to be processed.
  • the server may obtain the intersection between the element area and the target element envelope area, and determine the intersection between the element area and the target element envelope area as the final target element area outline.
  • the contour of the target element area combines the detection results of the element area detection of the image to be processed and the detection results of the target element area based on artificial intelligence in the image to be processed, which can effectively reduce the influence of errors in a single detection method, and the contour detection and division of the target element area is accurate. high degree.
  • the target element area in the image to be processed is detected by the target element area detection method based on artificial intelligence, and the envelope area of the target element area is searched for the envelope area of the target element, and then fused through the element area.
  • the determined element area and the target element envelope area are detected by the area, and the outline of the target element area is obtained.
  • the detection results of the target element area detection method of artificial intelligence can be effectively used to obtain the accurate target element envelope area, and at the same time, by fusing the images to be processed
  • the element area and the target element envelope area in the element area can be corrected by using the result of the element area detection to correct the target element envelope area, which improves the accuracy of detecting the outline of the divided target element area.
  • the process of element area detection that is, performing element area detection on the image to be processed to determine the element area in the image to be processed includes:
  • Step 302 Determine the channel difference feature corresponding to each pixel in the image to be processed.
  • the element area detection is performed on the image to be processed by using the prior knowledge of the color feature of the element, so as to determine the element area in the image to be processed.
  • the elements and non-elements in the to-be-processed image have significantly different color differences.
  • the color characteristics of each pixel in the to-be-processed image may be Realize element area detection; for example, in the stained image of physiological slice, the stained element and the non-stained element have obvious color difference, and the detection of the stained element area can be realized according to the color characteristics of each pixel in the stained image.
  • the channel difference feature is used to describe the color feature of each pixel in the image to be processed, and can be specifically obtained by the server according to the difference between the channel values corresponding to each color channel of the pixel.
  • each pixel in the image to be processed corresponds to three color channels R, G, and B, respectively, and the channel difference feature can be based on the channels of the three color channels R, G, and B.
  • the difference between the values is obtained, for example, it can be obtained according to the largest difference among the channel values of the three color channels of R, G, and B.
  • the server may traverse the channel values of each pixel in the image to be processed corresponding to each color channel, and determine the channel difference feature of the corresponding pixel according to each channel value.
  • Step 304 color segmentation is performed on the image to be processed based on the channel difference feature corresponding to each pixel to obtain a color segmentation image.
  • the color segmentation image is obtained by performing color segmentation processing on the image to be processed according to the channel difference feature corresponding to each pixel. For example, the pixel value of each pixel of the to-be-processed image can be updated according to the channel difference feature corresponding to each pixel. Color segmentation is performed on the image to be processed by using the channel difference feature corresponding to each pixel point to obtain a color segmentation image.
  • the color segmentation image reflects the color characteristics of the image to be processed.
  • the obtained color segmentation image reflects that each pixel of the image to be processed is biased towards grayscale or biased towards color. If it is biased towards grayscale, the maximum difference in the channel values of each color channel is small, and the value of the channel difference feature is small. , then the corresponding pixels in the color segmentation image are biased towards black, otherwise the corresponding pixels in the color segmentation image are biased towards white, so that the color area and achromatic area in the image to be processed can be intuitively determined according to the color segmentation image.
  • the processed image is a dyed image obtained by dyeing elements, then the color area in the image to be processed is the element area, so that the effective detection of the element area can be realized by using prior knowledge.
  • the server after obtaining the channel difference feature corresponding to each pixel in the image to be processed, the server performs color segmentation on the image to be processed based on the difference feature of each channel.
  • the pixel value is updated to realize the color segmentation of the image to be processed by using the channel difference feature to obtain a color segmentation image.
  • Step 306 perform binarization segmentation on the color segmentation image to obtain element regions in the image to be processed.
  • the binarization segmentation is to further segment the color segmentation image through the binarization process, so as to determine the element area in the image to be processed.
  • a preset binarization segmentation threshold can be used to traverse the pixels in the color segmentation image to perform binarization processing, and the pixel value of each pixel in the color segmentation image can be binary mapped, for example, it can be mapped to white or Black, so that the element area in the image to be processed can be visually determined according to the binarization segmentation result.
  • the binarization segmentation threshold can be flexibly set according to actual needs.
  • it can be set to a certain pixel value, or it can be adaptively set according to the pixel value of each pixel in the color segmentation image, such as according to the pixel value of each pixel.
  • the mean value sets the corresponding binarization segmentation threshold to more accurately perform binary segmentation on the color segmentation image.
  • the server performs binarization segmentation on the color segmentation image.
  • the server can obtain a preset binarization segmentation threshold, or set it according to the channel difference feature corresponding to each pixel in the image to be processed.
  • the binarization segmentation threshold is used, and the color segmentation image is binarized and segmented by the binarization segmentation threshold, so as to obtain the element area in the to-be-processed image according to the binarization segmentation result.
  • the region of interest can be divided into the image to be processed in advance, for example, the region of interest with a size of 100*100 pixels is used to divide the image to be processed, and each region of interest corresponding to the image to be processed is obtained.
  • the element area is detected in the area to obtain the element area in each area of interest, and finally the element area in the image to be processed is obtained by splicing the element areas in each area of interest.
  • the color segmentation of the image to be processed is performed by using the prior knowledge of element color features, and the color segmentation image is further mapped and segmented in combination with the binarization segmentation.
  • the degree of contrast between the element area and the non-element area is increased by the binarization segmentation, which further improves the detection accuracy of the element area, so as to accurately determine the element area in the image to be processed.
  • determining the channel difference feature corresponding to each pixel in the image to be processed includes: determining the pixel value corresponding to each pixel in the image to be processed; obtaining each color channel value of the corresponding pixel according to the pixel value ; Determine the maximum channel difference between the color channel values, and obtain the channel difference feature according to the maximum channel difference.
  • the channel difference feature is obtained according to the maximum channel difference between the channel values corresponding to each color channel of each pixel in the image to be processed, and each pixel in the image to be processed is processed based on the maximum channel difference.
  • Pixel value mapping to achieve color segmentation of the image to be processed, and obtain a color segmentation image.
  • the color channel value of each pixel in the image to be processed corresponds to the value of the pixel in each color channel, and the value of the color channel value is related to the color space where the image to be processed is located. For example, if the image to be processed is an RGB image, the color space of the image to be processed includes R, G and B, correspondingly including 3 color channel values.
  • the pixel value of a certain pixel is (123, 15, 255), then The color channel values of this pixel are 123, 15, and 255, respectively.
  • the maximum channel difference refers to the maximum value of the difference between the color channel values corresponding to the pixel point. Specifically, the difference between each color channel value can be calculated in pairs, and the maximum value can be obtained according to the calculated difference. The maximum channel difference reflects the degree of difference between the pixel points in each color channel.
  • the server determines the pixel value corresponding to each pixel in the image to be processed. Specifically, the server may traverse each pixel in the image to be processed, read each pixel The pixel value of the pixel point. After obtaining the pixel value corresponding to each pixel point, the server analyzes the pixel value corresponding to each pixel point, and determines each color channel value of each pixel point. Generally, the pixel value of a pixel is obtained by combining the color channel values of each color channel in a certain order. Based on the combination rule of the pixel value and the color channel image, the color channel value corresponding to each color channel can be determined according to the pixel value.
  • the server calculates the difference of each color channel value two by two, and determines the maximum channel difference value of the pixel point according to the difference result.
  • the server may also determine the maximum value and the minimum value from the respective color channel values, and directly determine the maximum channel difference value of the pixel point according to the absolute value of the difference between the maximum value and the minimum value.
  • the server obtains the channel difference feature according to the obtained maximum channel difference. For example, the maximum channel difference can be directly used as the channel difference feature of the corresponding pixel.
  • color-segmenting the image to be processed based on the channel difference value corresponding to each pixel point to obtain a color-segmented image including: performing pixel value mapping on the image to be processed according to the maximum channel difference value corresponding to each pixel point to obtain color segmentation. image.
  • pixel value mapping refers to mapping the pixel value of each pixel in the image to be processed based on the maximum channel difference value, so as to perform color segmentation on the image to be processed according to the maximum channel difference value.
  • the corresponding channel difference can be determined according to the maximum channel difference value.
  • value pixel and update the pixel value of the corresponding pixel in the image to be processed according to the channel difference pixel, so as to realize the color segmentation of the image to be processed, after traversing all the pixels in the image to be processed, the color segmentation image is obtained.
  • the server After the server determines the maximum channel difference value corresponding to each pixel in the image to be processed, the server performs pixel value mapping on the pixel value of the corresponding pixel point in the image to be processed according to the maximum channel difference value corresponding to each pixel point, so that based on the maximum channel difference The value updates the pixel value of the corresponding pixel to realize the color segmentation of the image to be processed. For example, the server can generate channel difference pixels according to the maximum channel difference.
  • the maximum channel difference of a pixel in the image to be processed is 200, it can generate RGB channel difference pixels (200, 200, 200), and use The channel difference pixel is mapped to the pixel value of the pixel point, so as to map the pixel value of the pixel point through the maximum channel difference value, so as to realize the color segmentation of the image to be processed, and obtain a color segmentation image.
  • the channel difference feature is obtained according to the maximum channel difference between the channel values corresponding to each color channel of each pixel in the image to be processed, and each pixel in the image to be processed is processed based on the maximum channel difference.
  • Pixel value mapping can effectively use the color characteristics of each pixel to accurately segment the image to be processed, and ensure the accuracy of the color segmented image.
  • performing binarization segmentation on the color segmentation image to obtain element regions in the image to be processed includes: performing denoising processing on the color segmentation image to obtain a denoised image; obtaining a binarized segmentation threshold; The valued segmentation threshold and the pixel value corresponding to each pixel in the denoised image are binarized and segmented on the denoised image to obtain the element area in the image to be processed.
  • the color segmentation image is denoised, for example, the color segmentation image is blurred and denoised, so as to remove noise in the color segmentation image and further improve the reliability of the color segmentation image.
  • the binarization segmentation threshold can be preset according to actual needs, for example, it can be preset as a certain pixel value, or it can be preset according to the pixel value of each pixel in the color segmentation image.
  • Binarization segmentation is to further segment the color segmentation image through binarization processing. Specifically, each pixel point can be divided into binary value according to the size relationship between the binarization segmentation threshold and the pixel value corresponding to each pixel point in the denoised image. Then, the element area in the image to be processed is obtained according to the binarization segmentation.
  • the server when the color segmentation image is binarized, the server performs denoising processing on the color segmentation image. For example, the server can perform Gaussian blur denoising on the color segmentation image. Through Gaussian blur denoising, the intensity of a pixel can be reduced. Correlate with surrounding points, reduce the influence of mutation, and get a denoised image with no noise.
  • the server obtains the binarization segmentation threshold. Specifically, the preset binarization segmentation threshold can be queried, or the binarization segmentation threshold can be determined according to the pixel value of each pixel in the color segmentation image or the denoised image. For example, the binarization segmentation threshold may be determined according to the maximum channel difference corresponding to each pixel in the image to be processed.
  • the binarization segmentation threshold can be set according to each region of interest, for example, according to the maximum channel corresponding to each pixel in each region of interest The difference determines the binarization segmentation threshold, so as to flexibly set the binarized segmentation threshold of each region of interest, improve the adaptation of the binary segmentation threshold to each region of interest, improve the effectiveness of the binary segmentation, and ensure The division accuracy of the element area.
  • the server After obtaining the binarized segmentation threshold, the server performs binary segmentation on the denoised image based on the binarized segmentation threshold and the pixel value corresponding to each pixel in the denoised image.
  • the size relationship of the pixel values corresponding to each pixel in the denoised image is binarized and mapped to the pixel value corresponding to each pixel in the denoised image to realize the binarization segmentation of the denoised image, and obtain the element area in the to-be-processed image.
  • the denoised image is binarized and segmented based on the binarization segmentation threshold, so that the denoised image is subjected to the binarization segmentation threshold. Segmentation, accurately dividing the element area and non-element area in the image to be processed.
  • the denoised image is binarized and segmented to obtain an element area in the to-be-processed image, including: determining the denoised image pixel value corresponding to each pixel point in the denoised image; pixel binarization mapping is performed on the pixel value corresponding to each pixel point in the denoised image based on the binarization segmentation threshold, and the pixel mapping result corresponding to each pixel point in the denoised image is obtained; The pixel mapping result corresponding to each pixel in the denoised image obtains the element area in the image to be processed.
  • the pixel value corresponding to each pixel in the denoised image is subjected to pixel binarization mapping by using the binarization segmentation threshold, and the element region in the to-be-processed image is obtained according to the pixel mapping result corresponding to each pixel.
  • the server determines the pixel value corresponding to each pixel in the denoised image. For example, each pixel in the denoised image can be traversed to determine the pixel value corresponding to each pixel.
  • the server traverses each pixel in the denoised image, and performs pixel binarization mapping on the pixel value corresponding to each pixel in the denoised image based on the binarization segmentation threshold. Specifically, according to the binarization segmentation threshold and the pixel corresponding to each pixel The values are compared numerically, and the pixels whose pixel values are greater than the binarization segmentation threshold and those whose pixel values are not greater than the binarization segmentation threshold are mapped to different pixel values, such as black or white pixel values, respectively, Thus, the element pixels and non-element pixels in the image to be processed are intuitively divided, and the element area in the image to be processed can be determined according to each element pixel.
  • element pixels refer to pixels belonging to elements in the image to be processed
  • non-element pixels are pixels that do not belong to elements, that is, each pixel in the element area is an element pixel, and the pixels in the non-element area are all pixel points. is a non-element pixel.
  • the pixel values corresponding to each pixel in the denoised image are respectively subjected to pixel binarization mapping to obtain the pixel mapping result corresponding to each pixel in the denoised image.
  • the result of the mapping is the element area in the image to be processed.
  • the pixel mapped to the element pixel value in the pixel mapping result can be determined as the element pixel, and the pixel mapped to the non-element pixel value can be determined as the element pixel.
  • the pixel point is determined as a non-element pixel point. For example, when the element pixel value is white and the non-element pixel value is black, the pixels whose pixel mapping result is white in the denoised image are determined as element pixels, and the pixels whose pixel mapping result is black are determined as non-element pixels.
  • the element area in the image to be processed can be determined according to all the element pixel points in the area where the pixel mapping result is white, so as to achieve accurate segmentation of the element area.
  • using an artificial intelligence-based target element area detection method to detect the target element area in the image to be processed includes: querying a target element area detection model pre-trained by using training image data marked with the target element area; The target element region detection model detects the target element region of the image to be processed to obtain the target element region probability map; performs target element region determination on the target element region probability map, and obtains the target element region in the to-be-processed image according to the determination result.
  • the target element region detection model is obtained by pre-training the training image data marked with the target element region.
  • the target element region detection model can detect the target element region of the input image and output the target element region probability map.
  • the target element region probability map represents the probability that each pixel in the image to be processed is the target element. By judging the probability in the target element region probability map, the target element region in the to-be-processed image can be determined.
  • the server queries the pre-trained target element region detection model, and the target element region detection model is obtained by training the training image data annotating the target element region.
  • the training image data of the marked target element area can be directly marked based on the target elements with obvious features, and the marking is more accurate, which can ensure the training effect of the training image data.
  • the server detects the target element region of the image to be processed through the target element region detection model obtained by the query.
  • the to-be-processed image can be input into the target element region detection model for target element region detection, and the target element region detection model outputs the detection results. Include the target element area probability map.
  • the server judges the target element region probability map output by the target element region detection model to determine whether each pixel in the target element region probability map is a target element, and determines the target element region in the image to be processed according to the judgment result.
  • the target element region detection model obtained by pre-training the training image data marking the target element region performs target element region detection on the image to be processed, and determines the target element region probability map output by the target element region detection model, and determines The target element region in the image to be processed is detected by the pre-trained target element region detection model, which can effectively detect the target element region and identify the accurate target element region from the to-be-processed image.
  • the target element region detection model is used to detect the target element region of the image to be processed to obtain the target element region probability map.
  • the region is input to the target element region detection model to detect the target element region, and the region of interest probability map corresponding to each region of interest output by the target element region detection model is obtained.
  • Target element area probability map is used to detect the target element region of the image to be processed to obtain the target element region probability map.
  • target element region detection is performed on each region of interest by the target element region detection model, and the detection results corresponding to the regions of interest are spliced to obtain the target element region detection model.
  • the region of interest is obtained by dividing the image to be processed.
  • the image to be processed can be divided into regions according to a preset size, so that the image to be processed is divided into several regions of interest, which reduces the amount of data for single processing. Improve data processing efficiency.
  • the size of the region of interest can be set according to actual needs, and a fixed size can be directly set, such as 1000*1000 pixels.
  • the server when performing target element region detection on the image to be processed through the target element region detection model, the server obtains each region of interest obtained by the region division process of the image to be processed.
  • the region division of the region of interest can be realized when the element region detection of the image to be processed is performed, and then each region of interest obtained by region division during element region detection is directly obtained when the target element region detection of the to-be-processed image is performed.
  • the server inputs each region of interest into the target element region detection model to perform target element region detection, and obtains a probability map of each region of interest output by the target element region detection model.
  • the region of interest probability map records the probability that the pixels in the corresponding region of interest are detected as the target element region.
  • the server stitches the probability maps of the regions of interest corresponding to the regions of interest respectively to obtain the probability map of the target element region corresponding to the image to be processed, so as to realize the detection of the target element region of the image to be processed.
  • target element region detection is performed on each region of interest obtained by the region division process of the image to be processed through the target element region detection model, and the detection results corresponding to each region of interest are spliced to obtain the target corresponding to the image to be processed.
  • the element region detection result can reduce the amount of data for a single target element region detection process, and can perform parallel processing on each region of interest, which can effectively improve the processing efficiency of the target element region detection.
  • the target element region determination is performed on the target element region probability map, and the target element region in the to-be-processed image is obtained according to the determination result, including: obtaining a probability threshold; based on the probability threshold and each pixel point in the target element region probability map For the corresponding probability value, perform probability binarization mapping on the probability map of the target element region, and obtain the probability mapping result of each pixel in the probability map of the target element region; according to the probability mapping result of each pixel in the probability map of the target element region, the probability map to be processed is obtained.
  • the target element area in the image is obtained.
  • the probability threshold is preset according to actual needs, for example, it can be set to 0.5, the probability threshold is used to determine the target element region of the target element region probability map, and determine the target element region in the image to be processed according to the target element region determination.
  • the server obtains a preset probability threshold, and the probability threshold is used to binarize the probability value of each pixel in the target element region probability map.
  • the server performs probability binarization mapping on the probability map of the target element region based on the probability threshold and the probability value corresponding to each pixel in the probability map of the target element region.
  • the server can compare the probability corresponding to each pixel in the probability map of the target element region.
  • each pixel is subjected to probability binarization mapping to obtain the probability mapping result of each pixel in the probability map of the target element area.
  • the probability threshold is 0.5. If the probability value of pixel A in the probability map of the target element area is 0.8 and the probability value of pixel B is 0.2, then the pixel value of pixel A is mapped to the white pixel value, and the pixel value of pixel A is mapped to the white pixel value. The pixel value of , is mapped to the black pixel value, so as to realize the binarization mapping of the pixel point A and the pixel point B. After traversing all the pixel points, the binarization mapping of the probability map of the target element region is realized.
  • the server After obtaining the probability mapping result of each pixel in the probability map of the target element area, the server obtains the target element area in the to-be-processed image based on each probability mapping result. For example, the server can map the target element pixel value according to each probability mapping result.
  • the pixel point is determined as the target element pixel point, and the target element area in the to-be-processed image is determined according to all the target element pixel points.
  • the probability binarization mapping is performed on the probability map of the target element region through a preset probability threshold, so as to determine the target element region in the image to be processed according to the probability mapping result of each pixel in the probability map of the target element region, and the The binarization map can intuitively divide the target element area and the non-target element area in the image to be processed accurately, which ensures the accuracy of the target element area division.
  • finding an envelope for the detected target element area to generate the target element envelope area includes: performing area connectivity on the target element area in the image to be processed to generate the target element connected area; removing the target element connected area The non-target element area in the area is obtained to obtain the updated target element connected area; the morphological expansion of the updated target element connected area is performed to obtain the target element envelope area.
  • the non-target elements in the connected area are removed, so as to filter the non-target elements within the range of the connected area, and then use morphological expansion to filter the edges of the connected area. Smoothing is carried out, and at the same time, the denoising of the smaller area of the connected area is realized, so as to realize the envelope search for the detected target element area, and obtain the target element envelope area with high recognition and division accuracy.
  • the server after detecting the target element area in the to-be-processed image, performs area connectivity on the target element area in the to-be-processed image. Specifically, the target element area and the non-target area in the to-be-processed image can be traversed through a filter of a preset size. area, so as to connect the target element area in the image to be processed through the filter to generate the target element connected area. The server then filters out the non-target elements within the range of the generated connected area of the target element, so as to ensure that all the target elements are inside the connected area of the target element. Specifically, the server removes the non-target element area in the target element connected area.
  • the server can directly update the non-target element area in the target element connected area to the target element area to obtain the updated target element connected area.
  • the interior of the updated target element connected region is the target element region.
  • the server performs smoothing processing on the edge of the updated connected area of the target element, and specifically obtains the envelope area of the target element by performing morphological expansion on the updated connected area of the target element.
  • performing area connectivity on the target element area in the image to be processed to generate the target element connected area includes: filtering and dividing the target element area and the non-target area in the image to be processed to obtain each filtered area; The pixel values of the pixel points in the filtering area are analyzed by area type, and the analysis result is obtained; the filtering area whose analysis result is a connected area type is connected to generate the connected area of the target element.
  • the area type analysis is performed on each filtering area respectively, and whether the filtering area is a connected area type is determined based on the analysis result, and the The filter area of the connected area type is connected to generate the connected area of the target element.
  • the server when connecting the target element regions in the image to be processed, filters and divides the target element regions and non-target regions in the image to be processed to obtain each filter region.
  • the size of the filter division can be determined according to the size of the filter. For example, when the size of the filter is 10*10 pixels, the target element area and the non-target area in the image to be processed are divided according to the filter to obtain each 10*10 Pixel-sized filter area. After each filtering area is determined, the server performs area type analysis on the pixel values of the pixels in each filtering area.
  • the server can traverse each filtering area, and for each pixel in each filtering area, if it detects that there is a target element pixel point, that is, there are target element pixels in the filtering area, then the type of the filtering area is determined to be a connected area type, that is, the filtering area can be identified as the area of the target element area, that is, the filtering area can be connected.
  • the server connects the filter regions whose corresponding analysis results are connected region types in each filter region to obtain a connected region of the target element.
  • the server when connecting the filter regions of each connected region type, maps the pixel values of the pixels in the filter regions of each connected region type to the pixel value of the target element, and then maps each connected region type after the mapping process.
  • the filter regions are connected to generate a connected region of the target element.
  • the image to be processed can be traversed directly through the set filter.
  • the filter area is determined as the type of connected area, and the pixel value of each pixel in the connected area is mapped to the target element. The pixel value corresponding to the pixel point.
  • the filter detects a white pixel in the filtering area, all the pixels in the filtering area are mapped to white pixels, that is, the filter is determined.
  • the area is the area that needs to be connected. After the filter traverses the to-be-processed image, the filter regions of each connected region type are connected to generate the target element connected region.
  • the artificial intelligence-based image processing method further includes: superimposing the outline of the target element area on the image to be processed, and marking the outline of the target element area in the image to be processed by a predetermined identification method.
  • the identification method can be flexibly set in advance according to actual needs, such as outline lines and symbols of highlighted colors. Specifically, after dividing and detecting the outline of the target element area from the image to be processed, the server superimposes the outline of the target element area into the image to be processed, and identifies the outline of the target element area in the image to be processed by a predetermined identification method, Therefore, the contour of the target element region is visually identified in the image to be processed, which facilitates subsequent image analysis based on the contour of the target element region and the to-be-processed image.
  • the present application also provides an application scenario where the above-mentioned artificial intelligence-based image processing method is applied.
  • the application of the artificial intelligence-based image processing method in this application scenario is as follows:
  • the images to be processed are landscape images, scene images, etc., where the elements are landscape objects in the landscape, such as trees, snow, sand and gravel, etc., or the elements are characters or animals in the scene image;
  • the target element is the number or number of different species. Different parts, or different characters or animals, such as men, women, children or the elderly, etc., the target elements can also be animals of different species such as dogs and cats in the scene image, and the target elements can also be characters or animals. Different parts, such as human hair, face and other parts, or animal torso, limbs or head, etc.:
  • the target element is determined according to the outline of the area to be detected.
  • the server obtains the captured landscape image, performs element area detection on the landscape image, and determines the tree area in the landscape image.
  • the target element area detection method based on artificial intelligence is used to detect the leaf area in the landscape image, and find the envelope of the detected leaf area, generate the leaf envelope area, and fuse the tree area and the leaf envelope area.
  • the leaf area contour is obtained, which intuitively shows the distribution of the leaf area contour in the landscape image, which is convenient for subsequent image analysis.
  • the present application further provides an application scenario where the above-mentioned artificial intelligence-based image processing method is applied.
  • the application of the artificial intelligence-based image processing method in this application scenario is as follows:
  • the image to be processed is a WSI (Whole Slide Image, full-field digital pathological slice) stained image
  • the elements are cells in the slice
  • the target elements are different types of cells, such as white blood cells, T cells or cancer cells, etc. .
  • the WSI staining image is a slice image based on the breast pathology PD-L1 (programmed cell death-Ligand 1, programmed cell death ligand 1, which is a ligand of PD-1).
  • PD-L1 programmed cell death-Ligand 1, programmed cell death ligand 1, which is a ligand of PD-1.
  • the number of immune cells can be obtained by the PD-L1 method.
  • the percentage of tumor area which is used to select the medication method for cancer treatment.
  • there are currently only qualitative methods for estimating the number of immune cells but no quantitative methods, resulting in inaccurate estimates of the IC ratio.
  • the immunohistochemical method to evaluate the immune efficacy based on PD-1 (programmed death 1, programmed death molecule receptor 1, which is a receptor protein on the surface of immune cells T cells)/PD-L1 has become the focus of research and hot spots. Changes in PD-L1 expression are associated with tumor progression and poor prognosis, and PD-L1 is considered to be an effective biomarker for predicting prognosis.
  • PD-1 programmed death 1, programmed death molecule receptor 1, which is a receptor protein on the surface of immune cells T cells
  • PD-L1 is considered to be an effective biomarker for predicting prognosis.
  • Breast cancer is the most common malignant tumor in women, and precision medicine provides an opportunity for more refined and individualized treatment of breast cancer.
  • paclitaxel protein binding agents have been proposed for the treatment of unresectable locally advanced TNBC (Triple-Negative Breast Cancer, triple negative breast cancer) or a diagnosis and treatment method for metastatic TNBC.
  • TNBC Triple-Negative Breast Cancer, triple negative breast cancer
  • FIG 4 the performance of tumors in 5 HE-stained images is included.
  • Figure A the HE-stained image is magnified 10 times for the local observation image. It can be seen that the sliced tissue can be divided into the surrounding stroma and the intratumoral stroma in the tumor area after staining.
  • Figure B, Figure C, Figure D, and Figure E are the results of tissue staining in different sections. The distribution of tumors can be determined through the HE staining images for subsequent diagnosis and treatment analysis.
  • Roche has proposed a guide method for interpreting IC value based on SP142 stain. Specifically, first observe the WSI image of HE staining (hematoxylin-eosin staining, hematoxylin-eosin staining), obtain the tumor area in the HE staining image, and then observe The results corresponded to the staining images of PD-L1, and the IC score was further determined by combining the estimated HE staining images and PD-L1 staining images.
  • the HE-stained WSI image was first observed to determine the tumor area, and then the observation results of the HE-stained WSI image were mapped to the PD-L1-stained WSI image, and the IC was estimated by combining the two. Percentage value for image analysis. Specifically, when observing the HE-stained WSI image, it is necessary to determine whether there is tumor and necrosis in the HE-stained WSI image, and to ensure adequacy, at least 50 viable tumor cells and stroma need to be detected, and the tumor area can also be estimated. area.
  • the present application provides an image processing method based on artificial intelligence to be applied in this scene, which can effectively and accurately divide and detect the contour of the target element area, which is convenient for subsequent image analysis and processing of the IC ratio value.
  • the server obtains a stained image, and the stained image is an image obtained after the WSI image is stained with PD-L1 dye.
  • the server performs tissue cell detection and cancer cell area detection on the stained image respectively, and obtains
  • the server fuses the sliced tissue area and the cancer cell area to determine the contour of the tumor area, and superimposes the contour of the tumor area into the stained image, so that the IC ratio can be estimated directly based on the superimposed result. There is no need to compare two stained images for analysis, which can effectively ensure the accuracy of IC ratio estimation.
  • the non-tissue area is more grayscale, and the tissue area has obvious staining. Therefore, if a pixel is judged to be closer to gray or closer to color, it can be judged that the Whether the pixel belongs to the background area or to the stained tissue cells.
  • the stained image can be divided into regions to obtain each ROI (Region of Interest), and each ROI can be traversed one by one to achieve tissue region detection.
  • the local threshold is calculated by using the pixel color difference to realize the segmentation of the background area and the tissue area based on the current ROI, thereby segmenting the slice tissue area from the stained image.
  • the slice tissue area detection is as follows (1),
  • Abs ⁇ is used to determine the absolute value of the difference between the maximum value and the minimum value.
  • HSV Human-Saturation-Value, Hue-Saturation
  • lightness-lightness color space.
  • the color segmentation image Diff roi (x, y) on the right is obtained after color segmentation is performed on the dyed image ROI (x, y) on the left.
  • gray The pixel area is the slice tissue area.
  • the color segmentation image is obtained, the color segmentation image is further binarized and segmented, so as to determine the element area in the image to be processed.
  • the specific processing is as follows (2),
  • the slice tissue in the stained image can be determined.
  • FIG. 11 it is the binarized segmentation result obtained by performing the binarization segmentation process on the color segmentation image in the embodiment shown in FIG. 10 , wherein the white area is the slice tissue area.
  • FIG. 12 is a stained image
  • FIG. 13 is a slice tissue area (white area) determined after slice tissue detection is performed on the stained image of FIG. 12 .
  • the network model was trained using 900 labeled images (832*832 pixels, 0.848 ⁇ m/pixel). In specific applications, if there is annotated data for the tumor cell area, the model can be directly trained to detect the tumor area, but in actual engineering, it is very difficult to obtain sufficient annotation, and the tumor area actually contains a variety of complex pathological tissues , even with labeling, it is difficult to train the deep learning network, that is, it is difficult to obtain the actual available tumor area.
  • the trained Linknet network model is used to predict the ROT of the stained images one by one, and then spliced to obtain a probability map of predicted cancer cell regions, the value of which is 0 to 1. Based on the cancer cell region probability map, a probability threshold of 0.5 is used to perform binarization mapping to obtain the binarized cancer cell region results. As shown in FIG. 15 , it is a cancer cell region probability map obtained by predicting the cancer cell region of the stained image based on the Linknet network model in one embodiment; as shown in FIG. The result of the cancer cell area obtained after the value mapping process. Among them, the white area is the cancer cell area.
  • the envelope of the entire cancer cell region is searched, that is, the tumor region is determined.
  • finding the envelope of the cancer cell area it is realized by calculating the connected area.
  • a filter of 10*10 pixels is used to traverse the images of the detected cancer cell area one by one. Within the range of 10*10 pixels, as long as there are pixels larger than 0 If pixels exist (white pixels), the entire 10*10 pixel ROI is regarded as a connected area.
  • a filter of 10*10 pixels is used on the scaled image of about 50 ⁇ m/pixel.
  • a 4*4 circular kernel kernel is used to achieve morphological expansion to obtain smooth edges, and at the same time, denoising is performed for smaller areas, which further improves the accuracy of the overall contour of the cancer area.
  • Figure 17 it is the connected domain detection result obtained after performing regional connectivity for the cancer cell region shown in Figure 16;
  • Figure 18 is the removal result after removing black holes for the connected domain detection result in Figure 17;
  • Figure 19 is for Figure 18 shows the overall contour results of the cancer region obtained after morphological expansion of the removal results.
  • the slice tissue area and the overall contour result of the cancer area are fused.
  • the tumor area can be obtained by calculating the intersection of the two.
  • the tumor area refers to the area of cancer cells, and part of the stroma between the areas of cancer cells.
  • Slice tissue area detection can obtain the area of all tissues, including tumor area and non-tumor area. Using cancer cell area detection, non-tumor area can be determined, and the detection result of tumor area can be obtained by combining these two results.
  • the outline of the cancer cell area obtained by fusing the cancer cell area in FIG. 18 and the sectioned tissue area in FIG. 16 is the result of the tumor area.
  • the tumor area was superimposed on the original stained image to mark the detected tumor area directly in the stained image.
  • the edge of the tumor area is relatively close, and the stained area of the non-tumor area is also well removed.
  • steps in the flowcharts of FIGS. 2-3 are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIG. 2 to FIG. 3 may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but may be executed at different times. The order of execution is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or stages within the other steps.
  • an image processing apparatus 2200 based on artificial intelligence is provided.
  • the apparatus can use software modules or hardware modules, or a combination of the two to become a part of computer equipment. It includes: image acquisition module 2202, element region detection module 2204, target element region detection module 2206, envelope generation module 2208 and region fusion module 2210, wherein:
  • An image acquisition module 2202 configured to acquire an image to be processed
  • the element area detection module 2204 is used to detect the element area of the image to be processed to determine the element area in the image to be processed;
  • the target element area detection module 2206 is used to detect the target element area in the image to be processed by adopting the target element area detection method based on artificial intelligence;
  • An envelope generation module 2208 for finding an envelope for the detected target element region, and generating the target element envelope region
  • the area fusion module 2210 is used to fuse the element area and the target element envelope area to obtain the outline of the target element area.
  • the target element region detection module detects the target element region in the to-be-processed image through the target element region detection method based on artificial intelligence, and the envelope generation module searches for the envelope of the target element region to obtain the target element envelope Then, the region fusion module fuses the element region and the target element envelope region determined by the element region detection module through the element region detection to obtain the target element region outline.
  • the detection results of the target element area detection method of artificial intelligence can be effectively used to obtain the accurate target element envelope area, and at the same time, by fusing the images to be processed
  • the element area and the target element envelope area in the element area can be corrected by using the result of the element area detection to correct the target element envelope area, which improves the accuracy of detecting the outline of the divided target element area.
  • the element area detection module 2204 includes a channel difference feature determination module, a color segmentation module and a binarization segmentation module; wherein: a channel difference feature determination module is used to determine the corresponding pixel points in the image to be processed respectively The channel difference feature of the Value segmentation to obtain the element area in the image to be processed.
  • the channel difference feature determination module includes a pixel value determination module, a channel value determination module, and a maximum channel difference determination module; wherein: the pixel value determination module is used to determine the pixel corresponding to each pixel in the image to be processed. The channel value determination module is used to obtain each color channel value of the corresponding pixel point according to the pixel value; the maximum channel difference determination module is used to determine the maximum channel difference value between the color channel values, and obtain according to the maximum channel difference value.
  • the channel difference feature; the color segmentation module is also used to perform pixel value mapping on the image to be processed according to the maximum channel difference corresponding to each pixel point to obtain a color segmentation image.
  • the binarization segmentation module includes a denoising module, a segmentation threshold acquisition module and a segmentation processing module; wherein: a denoising module is used to denoise the color segmentation image to obtain a denoised image; segmentation threshold acquisition The module is used to obtain the binarized segmentation threshold; the segmentation processing module is used to perform binary segmentation on the denoised image based on the binarized segmentation threshold and the pixel value corresponding to each pixel in the denoised image to obtain the image to be processed element area in .
  • the segmentation processing module includes a pixel value analysis module, a binarization mapping module, and a mapping result analysis module; wherein: a pixel value analysis module is used to determine a pixel value corresponding to each pixel in the denoised image; a binary value The mapping module is used to perform pixel binarization mapping on the pixel value corresponding to each pixel in the denoised image based on the binarization segmentation threshold, and obtain the pixel mapping result corresponding to each pixel in the denoised image; the mapping result analysis module , which is used to obtain the element area in the image to be processed according to the pixel mapping result corresponding to each pixel in the denoised image.
  • the target element region detection module 2206 includes a model query module, a model processing module and a model result processing module; wherein: a model query module is used to query the target element region pre-trained by using the training image data marked with the target element region The detection model; the model processing module is used to detect the target element area of the image to be processed through the target element area detection model to obtain the target element area probability map; the model result processing module is used to determine the target element area on the target element area probability map, The target element area in the image to be processed is obtained according to the determination result.
  • a model query module is used to query the target element region pre-trained by using the training image data marked with the target element region The detection model
  • the model processing module is used to detect the target element area of the image to be processed through the target element area detection model to obtain the target element area probability map
  • the model result processing module is used to determine the target element area on the target element area probability map, The target element area in the image to be processed is obtained according to the determination result.
  • the model processing module includes a region of interest module, a probability map obtaining module, and a probability map splicing module; wherein: the region of interest module is used to obtain each region of interest obtained from the image to be processed through region division processing; the probability The map acquisition module is used to input each region of interest into the target element region detection model to detect the target element region, and obtain the region of interest probability map corresponding to each region of interest output by the target element region detection model; the probability map splicing module, It is used to splicing the probability maps of each region of interest to obtain the probability map of the target element region corresponding to the image to be processed.
  • the model result processing module includes a probability threshold value acquisition module, a probability threshold value mapping module and a target element region module; wherein: a probability threshold value acquisition module is used to acquire a probability threshold value; a probability threshold value mapping module is used based on the probability threshold value and The probability value corresponding to each pixel in the probability map of the target element region is used to perform probability binarization mapping on the probability map of the target element region to obtain the probability mapping result of each pixel in the probability map of the target element region; The target element region in the image to be processed is obtained from the probability mapping result of each pixel in the target element region probability map.
  • the envelope generation module 2208 includes a region connectivity module, a region filling module, and a region expansion module; wherein: the region connectivity module is used to perform region connectivity on the target element region in the image to be processed to generate a target element connected region; The area filling module is used to remove the non-target element area in the connected area of the target element to obtain the updated connected area of the target element; the area expansion module is used to morphologically expand the connected area of the updated target element to obtain the target element envelope area.
  • the region connectivity module includes a filtering region obtaining module, a region type analysis module and a connectivity processing module; wherein: a filtering region obtaining module is used to filter and divide the target element region and the non-target region in the image to be processed, Obtain each filter area; the area type analysis module is used to analyze the area type based on the pixel values of the pixel points in each filter area, and obtain the analysis result; the connection processing module is used to connect the filter areas whose analysis result is a connected area type, Generates connected regions of target elements.
  • a superimposition processing module is further included, configured to superimpose the outline of the target element area into the image to be processed, and identify the outline of the target element area in the image to be processed by a predetermined identification method.
  • each module in the above-mentioned artificial intelligence-based image processing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device in one embodiment, the computer device may be a server, and its internal structure diagram may be as shown in FIG. 23 .
  • the computer device includes a processor, memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer device is used to store data.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer-readable instructions when executed by the processor, implement an artificial intelligence-based image processing method.
  • FIG. 23 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device comprising a memory and one or more processors, the memory having computer-readable instructions stored in the memory, the computer-readable instructions, when executed by the processor, cause one or more processing
  • the controller executes the steps in each of the above method embodiments.
  • one or more non-volatile readable storage media are provided that store computer-readable instructions that, when executed by one or more processors, cause the one or more processors to execute Steps in the above method embodiments.
  • a computer program product or computer program comprising computer readable instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computer device performs the steps in the foregoing method embodiments.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, or optical memory, and the like.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • the RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种基于人工智能的图像处理方法，由计算机设备执行。所述方法包括：获取待处理图像；对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；对所检测出的目标元素区域寻找包络，生成目标元素包络区域；融合元素区域和目标元素包络区域，获得目标元素区域轮廓。

Description

基于人工智能的图像处理方法、装置、计算机设备和存储介质
本申请要求于2020年11月02日提交中国专利局、申请号为2020112022810、发明名称为“基于人工智能的图像处理方法、装置和计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别是涉及一种基于人工智能的图像处理方法、装置、计算机设备和存储介质。
背景技术
计算机视觉(Computer Vision，CV)是使用计算机及相关设备对生物视觉的一种模拟，它的主要任务就是通过对采集的图片或视频进行处理以获得相应场景的三维信息，就像人类和许多其他类生物每天所做的那样。其中，物体识别和检测作为计算机视觉中非常基础且重要的一个研究方向。物体识别和检测，即给定一张输入图片，自动找出图片中的常见物体，并将其所属类别及位置输出出来，具体如人脸检测、车辆检测、组织检测等，在制造业、检验、文档分析、医疗诊断和军事等领域中得到了广泛的应用。
目前的物体识别和检测处理方法中，往往只是对单个对象进行识别检测，而对图像中包括目标对象区域轮廓无法进行有效的识别和划分，导致图像区域划分的准确度较低。
发明内容
根据本申请的各种实施例，提供一种基于人工智能的图像处理方法、装置、计算机设备和存储介质。
一种基于人工智能的图像处理方法，由计算机设备执行，所述方法包括：
获取待处理图像；
对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；
采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；
对所检测出的目标元素区域寻找包络，生成目标元素包络区域；及
融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
一种基于人工智能的图像处理装置，所述装置包括：
图像获取模块，用于获取待处理图像；
元素区域检测模块，用于对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；
目标元素区域检测模块，用于采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；
包络生成模块，用于对所检测出的目标元素区域寻找包络，生成目标元素包络区域；及
区域融合模块，用于融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
一种计算机设备，包括存储器和一个或多个处理器，所述存储器存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述一个或多个处理器执行以下步骤：
获取待处理图像；
对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；
采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；
对所检测出的目标元素区域寻找包络，生成目标元素包络区域；及
融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
一个或多个存储有计算机可读指令的非易失性可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
获取待处理图像；
对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；
采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；
对所检测出的目标元素区域寻找包络，生成目标元素包络区域；及
融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为一个实施例中基于人工智能的图像处理方法的应用环境图；
图2为一个实施例中基于人工智能的图像处理方法的流程示意图；
图3为一个实施例中元素区域检测的流程示意图；
图4为一个实施例中HE染色图像中肿瘤的示意图；
图5为传统IC占比值分析方法的流程示意图；
图6为一个实施例中HE染色WSI图像和PD-L1染色WSI图像的对比图；
图7为另一个实施例中HE染色WSI图像和PD-L1染色WSI图像的对比图；
图8为又一个实施例中HE染色WSI图像和PD-L1染色WSI图像的对比图；
图9为另一个实施例中基于人工智能的图像处理方法的流程示意图；
图10为一个实施例中色彩分割处理的结果示意图；
图11为图10所示实施例中二值化分割处理的结果示意图；
图12为一个实施例中的染色图像；
图13为图12所示实施例中检测出切片组织区域的结果示意图；
图14为一个实施例中的网络模型结构；
图15为一个实施例中模型输出的癌细胞区域概率图；
图16为图15实施例中判定得到的癌细胞区域结果示意图；
图17为图16实施例中的连通域检测结果示意图；
图18为图17实施例中去除黑洞后的去除结果示意图；
图19为图18实施例中形态学扩张后的结果示意图；
图20为图16和图18所示实施例中融合的结果示意图；
图21为图20实施例中叠加染色图像的结果示意图；
图22为一个实施例中基于人工智能的图像处理装置的结构框图；
图23为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例提供的方案涉及人工智能的计算机视觉技术，具体通过如下实施例进行说明。
本申请提供的基于人工智能的图像处理方法，由计算机设备执行，具体可以应用于如图1所示的应用环境中。其中，终端102通过网络与服务器104进行通信。终端102采集得到需要进行图像处理的待处理图像，如风景图像、场面图像、切片图像等各种场景中的图像，终端102将待处理图像发送至服务器104进行目标元素区域轮廓的检测划分处理。服务器104接收到待处理图像后，通过基于人工智能的目标元素区域检测方式检测出待处理图像中的目标元素区域，并对标元素区域寻找包络得到目标元素包络区域，再融合通过元素区域检测确 定的元素区域和目标元素包络区域，得到目标元素区域轮廓。此外，服务器104也可以从直接从数据库中获取待处理图像进行图像处理，以识别待处理图像中的目标元素区域轮廓。其中，终端102可以但不限于是具备拍摄功能各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，还可以为各种摄像设备或扫描设备等能够采集图像的设备，服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一个实施例中，如图2所示，提供了一种基于人工智能的图像处理方法，以该方法应用于图1中的服务器为例进行说明，包括以下步骤：
步骤202，获取待处理图像。
其中，待处理图像是需要进行区域检测划分的图像，待处理图像可以为在各种场景中拍摄或扫描的图像，如野外场景拍摄的风景图像、人群聚集场景拍摄的场面图像、医学场景中扫描的切片图像等各种类型的图像。待处理图像中包括拍摄或扫描抓取到的元素，如风景图像中拍摄到的树木、场景图像中拍摄的人等，且元素有不同的类别，如不同物种的树木、不同性别的人等，目前的图像识别中大多只是对图像中的元素进行判断，无法准确对不同类别元素的区域，而无法得到目标元素区域轮廓在图像中的分布，不利于后续基于待处理图像进行图像分析，如分析图像的树木中各种杨树的形状、场面图像中各男性的形体等。
具体地，待处理图像可以由拍摄设备拍摄得到，或由扫描设备扫描获得，并通过网络发送至服务器，由服务器对接收到的待处理图像进行目标元素区域轮廓的识别划分处理，从而确定待处理图像中目标元素区域轮廓的分布，便于后续的图像分析。在具体实现时，目标元素可以根据实际需要进行设置，如待处理图像中包括的元素为树木时，目标元素可以为设定的特定树木物种；又如待处理图像中包括的元素为细胞时，目标元素可以为特定类型的细胞，如白细胞、体细胞、癌细胞等。
步骤204，对待处理图像进行元素区域检测，以确定待处理图像中的元素区域。
其中，元素为待处理图像中拍摄或扫描抓取到的对象，而需要检测的目标元素的区域轮廓是元素中的某种类型，如树木为元素，则目标元素可以为树木的不同器官(如树叶，树干等)或不同物种类型的树木等；又如元素为狗，目标元素可以为狗的不同部位。元素区域指元素在待处理图像中分布的范围，如各种物种的树木在待处理图像中的分布范围。在具体应用时，元素根据图像处理的实际场景进行确定，如元素还可以为切片图像中的细胞，则元素区域为切片图像中的细胞区域，即切片组织区域。元素区域检测用于检测出待处理图像中的元素区域，具体可以通过各种对象检测算法，如R-CNN(Regions with Convolutional Neural Networks features，基于卷积神经网络特征的区域)算法、YOLO(You Only Look Once，只看一次)算法、SSD(Single Shot Detector，单镜头探测器)算法等对待处理图像进行元素区域检测，以确定待处理图像中的元素区域。此外，还可以利用先验知识，如利用待处理图像中元素和非元素的色彩特征先验知识，利用待处理图像中各像素点的色彩特征检测出待处理图像中的元素区域。
具体地，服务器获得待处理图像后，根据待处理图像中元素的类型特性确定对应的元素区域检测方式，通过该元素区域检测方式对待处理图像进行元素区域检测，以确定待处理图像中的元素区域。
步骤206，采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域。
其中，目标元素可以为元素中的子类别元素，即目标元素属于元素中需要针对检测出的元素。目标元素区域为目标元素在待处理图像的分布范围。目标元素区域检测方式基于人工智能实现，如可以为基于机器学习模型实现目标元素区域检测。例如，可以通过FCN(Fully Convolutional Networks，全卷积网络)模型、SegNet(A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation，一种用于图像分割的深度卷积编码－解码架构)模型、Linknet(Relational Embedding for Scene Graph，场景图的关系嵌入)模型等各种图像分割算法模型对目标元素区域进行检测，得到待处理图像中的目标元素区域。
具体地，在对待处理图像进行目标元素区域检时，服务器基于人工智能的目标元素区域检测方式，如采用预训练的检测模型对待处理图像进行目标元素区域检测方式，检测出待处理图像中的目标元素区域。
步骤208，对所检测出的目标元素区域寻找包络，生成目标元素包络区域。
目标元素包络区域为基于目标元素区域寻找包络而获得的初步的目标元素区域轮廓。具体地，在基于人工智能的目标元素区域检测方式检测出待处理图像中的目标元素区域后，服务器对该目标元素区域寻找包络，具体可以通过连通各目标元素区域，寻找到能够包括各目标元素区域的包络线，根据包络线包围的区域生成目标元素包络区域，通过对目标元素区域寻找包络，可以减小图像处理过程中误差的影响，得到准确的目标元素包络区域。
在具体实现时，服务器可以将目标元素区域进行连通，获得连通区域，再由服务器将连通区域的范围内的非目标元素区域去除，从而确保连通区域范围内均为准确的目标元素识别结果，最后由服务器对去除了非目标元素区域的连通区域进行平滑处理，如通过形态学扩张进行平滑处理，获得具有平滑边缘的目标元素包络区域，同时通过形态学扩张对较小的区域进行去噪处理，提高了目标元素包络区域识别划分的准确性。
步骤210，融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
得到元素区域和目标元素包络区域后，将二者融合，得到目标元素区域轮廓，其整体展现了目标元素在待处理图像中的分布范围。具体地，服务器可以求取元素区域和目标元素包络区域之间的交集，将元素区域和目标元素包络区域的交集确定为最终的目标元素区域轮廓。目标元素区域轮廓综合了待处理图像进行元素区域检测的检测结果和待处理图像基于人工智能的目标元素区域检测结果，能够有效减小单一检测方式中的误差影响，目标元素区域轮廓检测划分的准确度高。
上述基于人工智能的图像处理方法中，通过基于人工智能的目标元素区域检测方式检测出待处理图像中的目标元素区域，并对标元素区域寻找包络得到目标元素包络区域，再融合通过元素区域检测确定的元素区域和目标元素包络区域，得到目标元素区域轮廓。通过对基于人工智能的目标元素区域检测方式检测出目标元素区域寻找包络，可以有效利用人工智能的目标元素区域检测方式的检测结果，得到准确的目标元素包络区域，同时通过融合待处理图像中的元素区域和目标元素包络区域，可以利用元素区域检测的结果对目标元素包络区域进行校正，提高了检测划分的目标元素区域轮廓的准确度。
在一个实施例中，如图3所示，元素区域检测的处理过程，即对待处理图像进行元素区域检测，以确定待处理图像中的元素区域包括：
步骤302，确定待处理图像中各像素点分别对应的通道差值特征。
本实施例中，利用元素色彩特征的先验知识对待处理图像进行元素区域检测，从而确定待处理图像中的元素区域。具体地，待处理图像中的元素和非元素具有明显不同的颜色差异，如雪地场景的待处理图像中雪和非雪有明显颜色差异时，可以根据待处理图像中各像素点的颜色特征实现元素区域检测；又如生理切片的染色图像中，染色元素和非染色元素具有明显颜色差异，可以根据染色图像中各像素点的颜色特征实现染色元素区域的检测。
其中，通道差值特征用于描述待处理图像中各像素点的颜色特征，具体可由服务器根据像素点的各颜色通道对应通道值之间的差值得到。例如，待处理图像为RGB彩色图像时，待处理图像中每个像素点分别对应于R、G和B三个色彩通道，则通道差值特征可以根据R、G和B三个色彩通道的通道值之间的差值得到，如可以根据R、G和B三个色彩通道的通道值中最大差值得到。在具体应用时，服务器可以遍历待处理图像中各像素点对应于各色彩通道的通道值，并根据各通道值确定对应像素点的通道差值特征。
步骤304，基于各像素点分别对应的通道差值特征将待处理图像进行色彩分割，获得色彩分割图像。
其中，色彩分割图像根据各像素点对应的通道差值特征对待处理图像进行色彩分割处理得到，如可以根据各像素点对应的通道差值特征对待处理图像的各像素点的像素值进行更新， 实现利用各像素点对应的通道差值特征对待处理图像进行色彩分割，得到色彩分割图像。色彩分割图像反映了待处理图像的色彩特征，例如，若通道差值特征为各色彩通道的通道值中最大差值，色彩分割时直接将通道差值特征替换对应的像素点的像素值时，那么得到的色彩分割图像反映了待处理图像的各像素点偏向灰度或偏向彩色，如果偏向于灰度，则各色彩通道的通道值中最大差值较小，通道差值特征的数值较小，那么色彩分割图像中对应的像素点偏向于黑色，反之则色彩分割图像中对应的像素点偏向于白色，从而根据色彩分割图像可以直观确定待处理图像中的彩色区域和非彩色区域，若待处理图像是对元素进行染色得到的染色图像，那么待处理图像中的彩色区域即为元素区域，从而利用先验知识实现对元素区域的有效检测。
具体地，获得待处理图像中各像素点对应的通道差值特征后，服务器基于各通道差值特征对待处理图像进行色彩分割，如服务器基于各通道差值特征对待处理图像中对应的像素点的像素值进行更新，以实现利用通道差值特征对待处理图像进行色彩分割，得到色彩分割图像。
步骤306，对色彩分割图像进行二值化分割，获得待处理图像中的元素区域。
其中，二值化分割为通过二值化处理将色彩分割图像进一步进行分割，从而确定待处理图像中的元素区域。具体地，可以通过预设的二值化分割阈值遍历色彩分割图像中的像素点进行二值化处理，将色彩分割图像中的各像素点的像素值进行二值映射，如可以映射为白色或黑色，从而可以根据二值化分割结果直观确定待处理图像中的元素区域。二值化分割阈值可以根据实际需要进行灵活设置，如可以设置为确定的像素值，也可以根据色彩分割图像中各像素点的像素值进行适应性设定，如根据各像素点的像素值的均值设定相应的二值化分割阈值，以更准确地对色彩分割图像进行二值化分割。
具体地，在得到色彩分割图像后，服务器对色彩分割图像进行二值化分割，如服务器可以获取预设的二值化分割阈值，或根据待处理图像中各像素点对应的通道差值特征设置二值化分割阈值，并通过该二值化分割阈值对色彩分割图像进行二值化分割，从而根据二值化分割结果得到待处理图像中的元素区域。
在具体应用时，可以预先对待处理图像进行感兴趣区域划分，如以大小为100*100像素的感兴趣区域划分待处理图像，获得待处理图像对应的各感兴趣区域，并分别对各感兴趣区域进行元素区域检测，得到各感兴趣区域内的元素区域，最后通过拼接各感兴趣区域内的元素区域，得到待处理图像中的元素区域。通过感兴趣区域对待处理图像进行划分后进行元素区域检测，可以减少单次元素区域检测处理的数据量，而且可以对各感兴趣区域进行并行处理，能够有效提高元素区域检测的处理效率。
本实施例中，利用元素色彩特征的先验知识对待处理图像进行色彩分割，并结合二值化分割进一步对色彩分割图像进行映射分割，可以在有效利用先验知识进行准确元素区域检测的同时，通过二值化分割增加元素区域和非元素区域的对比程度，进一步提高了元素区域检测的准确性，从而准确确定待处理图像中的元素区域。
在一个实施例中，确定待处理图像中各像素点分别对应的通道差值特征，包括：确定待处理图像中各像素点分别对应的像素值；根据像素值得到相应像素点的各色彩通道值；确定各色彩通道值之间的最大通道差值，并根据最大通道差值得到通道差值特征。
本实施例中，根据待处理图像中每个像素点对应于各色彩通道的通道值之间的最大通道差值得到通道差值特征，并基于该最大通道差值对待处理图像中各像素点进行像素值映射，以实现对待处理图像的色彩分割，得到色彩分割图像。其中，待处理图像中各像素点的色彩通道值对应于像素点在每个色彩通道的数值，色彩通道值的取值与待处理图像所处的色彩空间相关。例如，待处理图像为RGB图像，则待处理图像的色彩空间包括R、G和B，对应包括3个色彩通道值，如对于某一像素点的像素值为(123，15，255)，则该像素点的色彩通道值分别为123、15和255。最大通道差值指像素点对应的各色彩通道值之间差值的最大值，具体可以将各色彩通道值进行两两求差，根据计算得到的差值寻找最大值得到。最大通道差 值反映了像素点在各色彩通道的差异程度，最大通道差值越大，则像素点对应于各色彩通道的差异越大，像素点的色彩越丰富，该像素点偏向于彩色；若最大通道差值越小，则像素点对应于各色彩通道的差异越小，像素点的色彩不丰富，该像素点偏向于灰度。
具体地，在确定待处理图像中各像素点对应的通道差值特征时，服务器确定待处理图像中各像素点对应的像素值，具体可以由服务器遍历待处理图像中各像素点，读取各像素点的像素值。得到各像素点对应的像素值后，服务器对各像素点对应的像素值进行解析，确定各像素点的各色彩通道值。一般地，像素点的像素值由各色彩通道的色彩通道值按照一定顺序组合得到，基于像素值和色彩通道图像的组合规则，可以根据像素值确定对应于各色彩通道的色彩通道值。服务器将各色彩通道值两两进行求差，根据求差结果确定该像素点的最大通道差值。此外，服务器也可以分别从各色彩通道值中确定最大值和最小值，根据该最大值和最小值的差值的绝对值直接确定该像素点的最大通道差值。服务器根据获得的最大通道差值得到通道差值特征，如可以直接将最大通道差值作为对应像素点的通道差值特征。
进一步地，基于各像素点分别对应的通道差值特征将待处理图像进行色彩分割，获得色彩分割图像，包括：按照各像素点对应的最大通道差值对待处理图像进行像素值映射，获得色彩分割图像。
其中，像素值映射指基于最大通道差值对待处理图像中各像素点的像素值进行映射处理，以根据最大通道差值对待处理图像进行色彩分割，如可以根据最大通道差值确定对应的通道差值像素，并根据通道差值像素更新待处理图像中相应像素点的像素值，从而实现对待处理图像的色彩分割，在遍历待处理图像中的所有像素点后，得到色彩分割图像。
具体地，服务器确定待处理图像中各像素点对应的最大通道差值后，按照各像素点对应的最大通道差值对待处理图像中相应像素点的像素值进行像素值映射，以基于最大通道差值更新对应像素点的像素值，实现对待处理图像的色彩分割。例如，服务器可以根据最大通道差值生成通道差值像素，如待处理图像中某一像素点的最大通道差值为200，则可以生成RGB通道差值像素(200，200，200)，并将该通道差值像素映射为该像素点的像素值，以通过最大通道差值对像素点的像素值进行映射，实现对待处理图像的色彩分割，得到色彩分割图像。
本实施例中，根据待处理图像中每个像素点对应于各色彩通道的通道值之间的最大通道差值得到通道差值特征，并基于该最大通道差值对待处理图像中各像素点进行像素值映射，可以有效利用各像素点的色彩特征对待处理图像进行准确的色彩分割，确保色彩分割图像的准确度。
在一个实施例中，对色彩分割图像进行二值化分割，获得待处理图像中的元素区域，包括：对色彩分割图像进行去噪处理，得到去噪图像；获取二值化分割阈值；基于二值化分割阈值和去噪图像中各像素点对应的像素值，对去噪图像进行二值化分割，得到待处理图像中的元素区域。
其中，对色彩分割图像进行去噪，如对色彩分割图像进行模糊去噪，以去除色彩分割图像中的噪声，进一步提高色彩分割图像的可信度。二值化分割阈值可以根据实际需要预先设置，如可以预设为确定的像素值，也可以根据色彩分割图像中各像素点的像素值进行预先设定。二值化分割为通过二值化处理将色彩分割图像进一步进行分割，具体可以根据二值化分割阈值和去噪图像中各像素点对应的像素值之间的大小关系将各像素点进行二值化分割，从而根据二值化分割得到待处理图像中的元素区域。
具体地，在对色彩分割图像进行二值化分割时，服务器对色彩分割图像进行去噪处理，如服务器可以对色彩分割图像进行高斯模糊去噪，通过高斯模糊去噪可以让一个像素点的强度与周围的点相关，减小突变的影响，得到去除干扰的去噪图像。服务器获取二值化分割阈值，具体可以查询预先设定的二值化分割阈值，也可以根据色彩分割图像或去噪图像中各像素点的像素值确定二值化分割阈值。例如，可以根据待处理图像中各像素点对应的最大通道差值确定二值化分割阈值。此外，若待处理图像被划分为各感兴趣区域分别进行元素区域检测时，二值化分割阈值则可以根据各感兴趣区域分别进行设置，如根据各感兴趣区域中各像 素点对应的最大通道差值确定二值化分割阈值，从而对各感兴趣区域的二值化分割阈值进行灵活设置，提高二值化分割阈值对各感兴趣区域的适配，提高二值化分割的有效性，确保元素区域的划分准确度。得到二值化分割阈值后，服务器基于二值化分割阈值和去噪图像中各像素点对应的像素值对去噪图像进行二值化分割，如服务器可以根据二值化分割阈值和去噪图像中各像素点对应的像素值的大小关系，将去噪图像中各像素点对应的像素值进行二值化映射，实现对去噪图像的二值化分割，得到待处理图像中的元素区域。
本实施例中，对色彩分割图像进行去噪处理去除色彩分割图像中的噪声后，基于二值化分割阈值对去噪图像进行二值化分割，以通过二值化分割阈值将去噪图像进行分割，准确划分出待处理图像中的元素区域和非元素区域。
在一个实施例中，基于二值化分割阈值和去噪图像中各像素点对应的像素值，对去噪图像进行二值化分割，得到待处理图像中的元素区域，包括：确定去噪图像中各像素点对应的像素值；基于二值化分割阈值分别对去噪图像中各像素点对应的像素值进行像素二值化映射，获得去噪图像中各像素点对应的像素映射结果；根据去噪图像中各像素点对应的像素映射结果获得待处理图像中的元素区域。
本实施例中，通过二值化分割阈值对去噪图像中各像素点对应的像素值进行像素二值化映射，并根据各像素点对应的像素映射结果获得所述待处理图像中的元素区域。具体地，在对去噪图像进行二值化分割时，服务器确定去噪图像中各像素点对应的像素值，如可以遍历去噪图像中各像素点，确定各像素点对应的像素值。服务器遍历去噪图像中各像素点，基于二值化分割阈值对去噪图像中各像素点对应的像素值进行像素二值化映射，具体根据将二值化分割阈值与各像素点对应的像素值分别进行数值比较，将像素值大于二值化分割阈值的像素点，以及像素值不大于二值化分割阈值的像素点分别映射为不同像素值，如分别映射为黑色或白色的像素值，从而直观地将待处理图像中的元素像素点和非元素像素点进行划分，根据各元素像素点即可确定待处理图像中的元素区域。其中，元素像素点指待处理图像中属于元素的像素点，非元素像素点为不属于元素的像素点，即元素区域的各像素点均为元素像素点，而非元素区域中的像素点均为非元素像素点。在基于二值化分割阈值分别对去噪图像中各像素点对应的像素值进行像素二值化映射，获得去噪图像中各像素点对应的像素映射结果后，服务器根据各像素点对应的像素映射结果得到待处理图像中的元素区域。例如，在将去噪图像中各像素点对应的像素值二值化映射后，可以将像素映射结果中映射为元素像素值的像素点确定为元素像素点，而将映射为非元素像素值的像素点确定为非元素像素点。例如，在元素像素值为白色，非元素像素值为黑色时，将去噪图像中像素映射结果为白色的像素点确定为元素像素点，将像素映射结果为黑色的像素点确定为非元素像素点，根据像素映射结果为白色的区域中的所有元素像素点可以确定待处理图像中的元素区域，从而实现对元素区域的准确分割。
在一个实施例中，采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域，包括：查询采用标注目标元素区域的训练图像数据预训练的目标元素区域检测模型；通过目标元素区域检测模型对待处理图像进行目标元素区域检测，得到目标元素区域概率图；对目标元素区域概率图进行目标元素区域判定，根据判定结果得到待处理图像中的目标元素区域。
其中，目标元素区域检测模型通过标注目标元素区域的训练图像数据预训练得到，目标元素区域检测模型可以对输入的图像进行目标元素区域检测，输出目标元素区域概率图。目标元素区域概率图表征了待处理图像中各像素点为目标元素的概率，通过对目标元素区域概率图中的概率进行判定，可以确定待处理图像中的目标元素区域。
具体地，在对待处理图像进行目标元素区域检测时，服务器查询预训练的目标元素区域检测模型，目标元素区域检测模型通过标注目标元素区域的训练图像数据训练得到，相比于标注目标元素区域轮廓，标注目标元素区域的训练图像数据训练可以直接基于特征明显的目标元素进行标注，标注更准确，能够确保训练图像数据的训练效果。服务器通过查询得到的 目标元素区域检测模型对待处理图像进行目标元素区域检测，具体可以将待处理图像输入目标元素区域检测模型中进行目标元素区域检测，由目标元素区域检测模型输出检测结果，检测结果包括目标元素区域概率图。服务器对目标元素区域检测模型输出的目标元素区域概率图进行判定，以确定目标元素区域概率图中各像素点是否为目标元素，从而根据判定结果确定待处理图像中的目标元素区域。
本实施例中，通过标注目标元素区域的训练图像数据预训练得到的目标元素区域检测模型对待处理图像进行目标元素区域检测，并对目标元素区域检测模型输出的目标元素区域概率图进行判定，确定待处理图像中的目标元素区域，通过预训练的目标元素区域检测模型进行目标元素区域检测，可以实现对目标元素区域的有效检测，从待处理图像中识别出准确的目标元素区域。
在一个实施例中，通过目标元素区域检测模型对待处理图像进行目标元素区域检测，得到目标元素区域概率图，包括：获取待处理图像经过区域划分处理获得的各感兴趣区域；分别将各感兴趣区域输入目标元素区域检测模型进行目标元素区域检测，得到目标元素区域检测模型输出的各感兴趣区域分别对应的感兴趣区域概率图；将各感兴趣区域概率图进行拼接，得到待处理图像对应的目标元素区域概率图。
本实施例中，在将待处理图像划分成各感兴趣区域后，由目标元素区域检测模型分别对各感兴趣区域进行目标元素区域检测，并将各感兴趣区域对应的检测结果拼接，得到待处理图像对应的目标元素区域检测结果。其中，感兴趣区域通过对待处理图像进行区域划分得到，如可以按照预设的尺寸对待处理图像进行区域划分，从而将待处理图像划分为若干个感兴趣区域，降低了单次处理的数据量，提高数据的处理效率。在具体实现时，感兴趣区域的尺寸可以根据实际需要进行设置，可以直接设置固定的尺寸大小，如设置为1000*1000像素。
具体地，在通过目标元素区域检测模型对待处理图像进行目标元素区域检测时，服务器获取待处理图像经过区域划分处理获得的各感兴趣区域。其中，感兴趣区域的区域划分可以在对待处理图像进行元素区域检测时实现，则在对待处理图像进行目标元素区域检测时，直接获取元素区域检测时区域划分得到的各感兴趣区域。服务器将各感兴趣区域输入目标元素区域检测模型中进行目标元素区域检测，获得由目标元素区域检测模型输出的各感兴趣区域概率图。感兴趣区域概率图记录了对应感兴趣区域中的像素点检测为目标元素区域的概率。服务器将各感兴趣区域分别对应的感兴趣区域概率图进行拼接，得到待处理图像对应的目标元素区域概率图，从而实现对待处理图像的目标元素区域检测。
本实施例中，通过目标元素区域检测模型分别对待处理图像经过区域划分处理获得的各感兴趣区域进行目标元素区域检测，并将各感兴趣区域对应的检测结果拼接，得到待处理图像对应的目标元素区域检测结果，可以减少单次目标元素区域检测处理的数据量，而且可以对各感兴趣区域进行并行处理，能够有效提高目标元素区域检测的处理效率。
在一个实施例中，对目标元素区域概率图进行目标元素区域判定，根据判定结果得到待处理图像中的目标元素区域，包括：获取概率阈值；基于概率阈值和目标元素区域概率图中各像素点对应的概率值，对目标元素区域概率图进行概率二值化映射，获得目标元素区域概率图中各像素点的概率映射结果；根据目标元素区域概率图中各像素点的概率映射结果得到待处理图像中的目标元素区域。
其中，概率阈值根据实际需要预先设置，如可以设置为0.5，概率阈值用于对目标元素区域概率图进行目标元素区域判定，根据目标元素区域判定确定待处理图像中的目标元素区域。具体地，在得到目标元素区域检测模型输出的目标元素区域概率图后，服务器获取预设的概率阈值，概率阈值用于对目标元素区域概率图中各像素点的概率值进行二值化映射。服务器基于概率阈值和目标元素区域概率图中各像素点对应的概率值，对目标元素区域概率图进行概率二值化映射，具体可以由服务器分别比较目标元素区域概率图中各像素点对应的概率值和概率阈值的大小，根据比较结果将各像素点进行概率二值化映射，获得目标元素区域概率图中各像素点的概率映射结果。例如，概率阈值为0.5，若目标元素区域概率图中像素点 A的概率值为0.8，像素点B的概率值为0.2，则将像素点A的像素值映射为白色像素值，将像素点A的像素值映射为黑色像素值，从而实现对像素点A和像素点B的二值化映射，在遍历所有像素点后，实现对目标元素区域概率图的二值化映射。得到目标元素区域概率图中各像素点的概率映射结果后，服务器基于各概率映射结果得到所述待处理图像中的目标元素区域，如服务器可以根据各概率映射结果中映射为目标元素像素值的像素点确定为目标元素像素点，并根据所有目标元素像素点确定待处理图像中的目标元素区域。
本实施例中，通过预设的概率阈值对目标元素区域概率图进行概率二值化映射，以根据目标元素区域概率图中各像素点的概率映射结果确定待处理图像中的目标元素区域，通过对二值化映射可以直观对待处理图像中的目标元素区域和非目标元素区域进行准确划分，确保了目标元素区域划分的准确度。
在一个实施例中，对所检测出的目标元素区域寻找包络，生成目标元素包络区域，包括：对待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域；去除处于目标元素连通区域内的非目标元素区域，得到更新后的目标元素连通区域；对更新后的目标元素连通区域进行形态学扩张，获得目标元素包络区域。
本实施例中，将所检测出的目标元素区域进行区域连通后，去除连通区域内的非目标元素，从而对连通区域的范围内部进行非目标元素过滤，再通过形态学扩张对连通区域的边缘进行平滑处理，同时实现对连通区域边较小区域的去噪，从而实现了对所检测出的目标元素区域寻找包络，得到了识别划分准确性高的目标元素包络区域。
具体地，在检测出待处理图像中的目标元素区域后，服务器对待处理图像中的目标元素区域进行区域连通，具体可以通过预设尺寸的滤波器遍历待处理图像中的目标元素区域和非目标区域，以通过滤波器将待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域。服务器再对生成的目标元素连通区域范围内非目标元素进行滤除，从而确保目标元素连通区域内部均为目标元素。具体地，服务器去除处于目标元素连通区域内的非目标元素区域，如服务器可以直接将处于目标元素连通区域内的非目标元素区域更新为目标元素区域，得到更新后的目标元素连通区域。更新后的目标元素连通区域的内部均为目标元素区域。进一步地，服务器对更新后的目标元素连通区域的边缘进行平滑处理，具体通过对更新后的目标元素连通区域进行形态学扩张，得到目标元素包络区域。通过对更新后的目标元素连通区域进行形态学扩张，可以对更新后的目标元素连通区域的边缘进行平滑处理，同时针对更新后的目标元素连通区域边缘较小区域进行有效去噪，进一步提高了目标元素包络区域的识别划分准确度。
在一个实施例中，对待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域，包括：将待处理图像中的目标元素区域和非目标区域进行滤波划分，得到各滤波区域；基于各滤波区域中像素点的像素值进行区域类型分析，获得分析结果；将分析结果为连通区域类型的滤波区域进行连通，生成目标元素连通区域。
本实施例中，在将待处理图像中的目标元素区域和非目标区域划分为各滤波区域后，分别对各滤波区域进行区域类型分析，基于分析结果确定滤波区域是否为连通区域类型，并将连通区域类型的滤波区域进行连通，生成目标元素连通区域。
具体地，在连通待处理图像中的目标元素区域时，服务器对待处理图像中的目标元素区域和非目标区域进行滤波划分，得到各滤波区域。其中，滤波划分的大小可以根据滤波器的大小确定，如滤波器的大小为10*10像素时，则根据滤波器将待处理图像中的目标元素区域和非目标区域划分得到若各个10*10像素大小的滤波区域。确定各滤波区域后，服务器对各滤波区域中像素点的像素值进行区域类型分析，具体可以由服务器对遍历各滤波区域，针对每个中滤波区域的各像素点，若检测到存在目标元素像素点，即滤波区域中存在目标元素像素点，则确定该滤波区域的类型为连通区域类型，即可以将该滤波区域认定为目标元素区域的区域，即可以将该滤波区域进行连通。得到各滤波器的分析结果后，服务器将各滤波区域中对应分析结果为连通区域类型的滤波区域进行连通，得到目标元素连通区域。在具体实现 时，在连通各连通区域类型的滤波区域时，服务器将各连通区域类型的滤波区域中的像素点的像素值均映射为目标元素像素值，再将映射处理后的各连通区域类型的滤波区域进行连通，从而生成目标元素连通区域。
在具体应用中，可以直接通过设定的滤波器遍历待处理图像，对于滤波器处理各滤波区域的范围中，若存在目标元素像素点，如可以根据滤波器处理的范围中各像素点的像素值进行判断，若存在目标元素像素点对应的像素值，则认为存在目标元素像素点，将该滤波区域确定为连通区域类型，并将该连通区域内的各像素点的像素值映射为目标元素像素点对应的像素值。例如，若目标元素像素点对应的像素值为白色像素值，若滤波器在滤波区域中检测到白色像素点时，则将该滤波区域的所有像素点均映射为白色像素点，即确定该滤波区域为需要进行连通处理的区域。在滤波器遍历待处理图像处理完成后，连通各连通区域类型的滤波区域，生成目标元素连通区域。
在一个实施例中，基于人工智能的图像处理方法还包括：将目标元素区域轮廓叠加至待处理图像中，并通过预定的标识方式在待处理图像中标识出目标元素区域轮廓。
其中，标识方式可以根据实际需要预先灵活设置，如可以为高亮颜色的轮廓线条、符号等。具体地，在从待处理图像中划分检测到目标元素区域轮廓后，服务器将该目标元素区域轮廓叠加至待处理图像中，并通过预定的标识方式在待处理图像中标识出目标元素区域轮廓，从而在待处理图像中直观的标识出目标元素区域轮廓，便于后续基于目标元素区域轮廓和待处理图像进行图像分析。
本申请还提供一种应用场景，该应用场景应用上述的基于人工智能的图像处理方法。具体地，该基于人工智能的图像处理方法在该应用场景的应用如下：
待处理图像为风景图像、场面图像等，其中元素为风景中风景物体对象，如树木、雪、砂石等，或者元素为场景图像中的人物或动物；目标元素为不同物种的数目或数目的不同部位，或者为不同的人物或动物，如男性、女性、儿童或老人等，目标元素也可以为场面图像中的狗、猫等不同物种的动物，此外，目标元素也可以为人物或动物的不同部位，如人物的毛发、脸部等各种部位，又如为动物的躯干、四肢或头部等：目标元素根据需要检测的区域轮廓确定。
在一个具体应用中，服务器获取拍摄得到的风景图像，对风景图像进行元素区域检测，确定风景图像中的树木区域。另一方面，采用基于人工智能的目标元素区域检测方式，检测出风景图像中的树叶区域，并对所检测出的树叶区域寻找包络，生成树叶包络区域，融合树木区域和树叶包络区域，得到树叶区域轮廓，该树叶区域轮廓直观展示了风景图像中树叶区域轮廓的分布，便于后续图像分析。
本申请还另外提供一种应用场景，该应用场景应用上述的基于人工智能的图像处理方法。具体地，该基于人工智能的图像处理方法在该应用场景的应用如下：
本应用场景中，待处理图像为WSI(Whole Slide Image，全视野数字病理切片)染色图像，元素为切片中的细胞，目标元素为不同类型的细胞，如可以为白细胞、T细胞或癌细胞等。
在一具体的应用中，WSI染色图像为基于乳腺病理PD-L1(programmed cell death-Ligand 1，程序性死亡因子配体1，为PD-1的配体)方法的切片图像。在乳腺癌治疗中，用PD-L1方法可以获得免疫细胞数量(PD-L1的表达)，基于该免疫细胞数量进一步评估抵御癌症的能力，即通过估计染色的免疫细胞(IC，immune cell)在肿瘤区域占比数值，用于选择癌症治疗的用药方法。然而，对免疫细胞数量估计目前只有定性方法，没有定量方法，导致对于IC比例值估计不准确。
目前，免疫组织化学法评估基于PD-1(programmed death 1，程序性死亡分子受体1，它是免疫细胞T细胞表面的一种受体蛋白)/PD-L1的免疫疗效已成为研究的重点和热点。PD-L1表达变化与肿瘤进展和不良预后有关，PD-L1被认为是预测预后的有效生物标志物。乳腺癌是女性最常见的恶性肿瘤，精准医学则为乳腺癌更加精细化、个体化的治疗提供了契机。当 前已提出结合紫杉醇蛋白结合剂用于治疗无法切除的局部晚期TNBC(Triple-Negative Breast Cancer，三阴性乳腺癌)或转移性TNBC的诊疗方法。如图4所示，包括5种HE染色图像中肿瘤的表现情况。在图A中，为HE染色图像放大10倍后局部观测图像，可以看出，切片组织染色后可以划分成肿瘤周围间质和肿瘤区域中的肿瘤内间质。图B、图C、图D和图E为不同切片组织染色的结果，通过该HE染色图像可以确定肿瘤的分布情况，以便后续进行诊疗分析。罗氏公司提出了基于SP142染色剂判读IC值的指导方法，具体先观察HE染色(hematoxylin-eosin staining，苏木精-伊红染色法)WSI图像，获取HE染色图像中的肿瘤区域，再将观察结果对应到PD-L1的染色图像中，进一步结合估计HE染色图像和PD-L1染色图像确定IC占比值(IC score)。
如图5所示，传统的IC占比值分析过程中，先观察HE染色WSI图像，确定肿瘤区域，再将HE染色WSI图像的观察结果对应到PD-L1染色WSI图像中，结合二者估计IC占比值进行图像分析。具体地，在观察HE染色WSI图像时，需要确定HE染色WSI图像中是否存在肿瘤和坏死，同时为确保充分性，需要检测到至少50个活的肿瘤细胞及间质，此外还可以估计肿瘤区域的面积。由此可知，传统的IC占比值估计的处理方法中，需要2种不同的WSI染色图像，即需要做2个玻片，一个用HE染色剂染色，一个用PD-L1染色剂染色，明显增加了图像分析的工作量。而且，两种染色剂获得的图像难以保持完全一致的，因为制作2个不同染色的玻片,需要从蜡块上切下2片不同的癌症组织，从获取的WSI图像上会有误差，以及位置的平移、旋转等，影响IC占比值的估计。如图6-图8所示，为不同实施例中HE染色WSI图像和PD-L1染色WSI图像的对比，可以看出，图6中HE染色WSI图像和PD-L1染色WSI图像有一定程度的旋转，图7中HE染色WSI图像和PD-L1染色WSI图像产生了翻转，图8中HE染色WSI图像和PD-L1染色WSI图像有一定程度的旋转和平移。所以，从HE染色WSI图像上获取的肿瘤组织区域，并在PD-L1染色WSI图像上找到相同的区域，将肿瘤区域对应到PD-L1染色WSI图像进行IC占比值估计时，难以进行有效的对应，导致IC占比值估计的准确度大打折扣，导致图像分析的准确度较低。
基于此，本申请提供基于人工智能的图像处理方法应用于该场景中，可以有效准确对目标元素区域的轮廓进行划分检测，便于后续IC占比值的图像分析处理。具体地，如图9所示，服务器获取染色图像，染色图像为通过PD-L1染色剂对WSI图像进行染色处理后得到的图像，服务器对染色图像分别进行组织细胞检测和癌细胞区域检测，得到切片组织区域和癌细胞区域，服务器将切片组织区域和癌细胞区域进行融合后确定肿瘤区域轮廓，并将肿瘤区域轮廓叠加到染色图像中，以便后续可以直接根据叠加结果进行IC占比值估计，而不需要对比两份染色图像进行分析，可以有效确保IC占比值估计的准确度。
进一步地，对于切片组织区域检测，可以利用先验知识进行有效检测，即非组织区域更偏向灰度，组织区域存在明显染色，因此判断一个像素点更接近灰色或者更接近彩色，就可以判断该像素点是属于背景区域还是属于染色的组织细胞。而考虑到WSI图像的尺寸较大，可以先将染色图像进行区域划分，得到各ROI(Region of Interest，感兴趣区域)，通过逐个遍历处理每个ROI，实现组织区域检测。具体在每个ROI里，利用像素点色彩差值计算局部阈值，实现基于当前ROI的背景区域和组织区域的分割，从而从染色图像中分割出切片组织区域。对于每个ROI，切片组织区域检测如下式(1)，
Diff roi(x,y)＝Abs{Max[ROI(x,y,i)]| i＝1,2,3-Min[ROI(x,y,i)]| i＝1,2,3}  (1)
其中，Diff roi(x,y)指在ROI内的(x,y)点的不同的三个色彩通道(i＝1,2,3)中，像素值的最大值和最小值的差值。其中，Max[ROI(x,y,i)]| i＝1,2,3用于确定(x,y)点的三个色彩通道的最大值,Min[ROI(x,y,i)]| i＝1,2,3用于确定(x,y)点的三个色彩通道的最小值，Abs{}用于确定最大值和最 小值的差值的绝对值。本实施例中，图像的色彩通道(i＝1,2,3)是R，G和B三个通道，此外也可以是其他色彩空间的通道，如HSV(Hue-Saturation-Value，色调-饱和度-明度)色彩空间。一般地，如果某个像素点是彩色像素，则Abs{}的值较大；如果是灰度像素，则Abs{}的值相对较小。如图10所示，为一个实施例中，对左侧的染色图像ROI(x,y)进行色彩分割后得到右侧的色彩分割图像Diff roi(x,y)，在色彩分割图像中，灰色像素区域为切片组织区域。
得到色彩分割图像后，进一步对色彩分割图像进行二值化分割，从而确定待处理图像中的元素区域。具体处理如下式(2)，
Mask roi(x,y)＝Binary{GaussianBlur[Diff roi(x,y)]}| Thresh＝Diff(x,y)/10  (2)
其中，先对色彩分割图像Diff roi(x,y)进行模糊去噪，具体使用高斯模糊去噪GaussianBlur[]，然后，针对每个像素点进行二值化Binary{}计算，具体二值化分割阈值Thresh＝Diff(x，y)/10进行二值化分割，得到二值化分割结果Mask roi(x，y)，根据二值化分割结果可以确定染色图像中的切片组织。如图11所示，为图10所示实施例中对色彩分割图像进行二值化分割处理后得到的二值化分割结果，其中白色区域即为切片组织区域。
如图12-图13所示，在一个实施例中，图12为染色图像，图13为针对图12的染色图像进行切片组织检测后确定的切片组织区域(白色区域)。
另一方面，对于癌细胞区域检测处理，通过预训练的Linknet网络模型(Chaurasia2017)对染色图像进行癌细胞区域检测。如图14，Linknet网络模型结构中，卷积处理及通过conv进行卷积操作，转置卷积表示通过full-conv进行卷积操作。/2表示因子为2的下采样，使用带步长的卷积来实现。*2表示因子为2的上采样。在卷积层之间使用BN(Batch Normalization，是批标准化)和ReLU(Rectified Linear Unit，线性整流函数)。左半部分是编码器(Encoder Block)，右半部分是解码器(Decoder Block)。Encoder Block开始处有一个initial block(初始块)，其余部分由残差块组成。训练网络模型使用900标注图像(832*832像素，0.848微米/像素)。在具体应用时，若具有针对肿瘤细胞区域的标注数据，可以直接训练模型进行肿瘤区域检测，但实际工程中，足够的标注非常难获得，且肿瘤区域实际上包含了较为复杂的各种病理组织，即使进行标注，也比较难训练深度学习网络，即难以获得实际可用的肿瘤区域。基于此，考虑到癌细胞的特征是非常不同于其他病理组织，可以不对肿瘤区域进行标注，只需要对癌细胞区域进行标注，通过标注癌细胞区域的标注图像进行模型训练。在200多图像的测试集上，模型精度达到F1(F-measure，F值)＝0.89，Recall(召回率)＝0.89，Precision(准确率)＝0.91的水平时可以结束训练，得到训练完成的模型。
使用训练完成的Linknet网络模型对染色图像的ROT逐个进行预测，然后拼接，获得预测癌细胞区域概率图，其值是0～1。基于癌细胞区域概率图，使用概率阈值0.5进行二值化映射，获得二值化的癌细胞区域结果。如图15所示，为一个实施例中基于Linknet网络模型对染色图像进行癌细胞区域预测得到的癌细胞区域概率图；如图16所示，为使用概率阈值0.5 对癌细胞区域概率图进行二值化映射处理后得到的癌细胞区域结果。其中，白色区域为癌细胞区域。
进一步地，在得到切片组织区域和癌细胞区域的后处理中，考虑到连接癌细胞区域之间的间质，寻找整个癌细胞区域的包络，即确定肿瘤区域。在寻找癌细胞区域的包络时，通过计算连通区域实现，具体使用10*10像素的滤波器，逐个遍历检测出癌细胞区域的图像，在10*10像素范围内，只要有像素大于0的像素存在(白色像素点)，则整个10*10像素的ROI视为连通区域。其中，10*10像素的滤波器对应使用在约50微米/像素的缩放后的图像上。进一步地，使用4*4的圆形kernel(核)实现形态学扩张，获取平滑的边缘，同时针对较小的区域进行去噪，进一步提高了癌症区域整体轮廓的准确度。
如图17所示，为针对图16所示的癌细胞区域进行区域连通后得到的连通域检测结果；图18为针对图17中的连通域检测结果去除黑洞后的去除结果；图19为针对图18的去除结果进行形态学扩张后得到的癌症区域的整体轮廓结果。
在得到癌症区域的整体轮廓结果后，将切片组织区域和癌症区域的整体轮廓结果进行融合，具体可以通过计算二者的交集，得到肿瘤区域。肿瘤区域指的是癌细胞区域，以及癌细胞区域之间的部分间质。切片组织区域检测能获取全部组织的区域，这些区域包括了肿瘤区域和非肿瘤区域，利用癌细胞区域检测，可以判定非肿瘤区域，融合这两个结果可以得到肿瘤区域的检测结果。如图20，为融合图18中的癌细胞区域和图16中切片组织区域得到的癌细胞区域轮廓，即肿瘤区域的结果。进一步地，将肿瘤区域叠加到最初的染色图像中，以将检测出的肿瘤区域直接在染色图像中进行标记。如图21所示，为将图20中的肿瘤区域叠加到染色图像后进行标记的展示效果，可以看出，肿瘤区域边缘比较贴合，同时非肿瘤区域的染色区域也得到较好的去除。
应该理解的是，虽然图2-图3的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2-图3中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图22所示，提供了一种基于人工智能的图像处理装置2200，该装置可以采用软件模块或硬件模块，或者是二者的结合成为计算机设备的一部分，该装置具体包括：图像获取模块2202、元素区域检测模块2204、目标元素区域检测模块2206、包络生成模块2208和区域融合模块2210，其中：
图像获取模块2202，用于获取待处理图像；
元素区域检测模块2204，用于对待处理图像进行元素区域检测，以确定待处理图像中的元素区域；
目标元素区域检测模块2206，用于采用基于人工智能的目标元素区域检测方式，检测出待处理图像中的目标元素区域；
包络生成模块2208，用于对所检测出的目标元素区域寻找包络，生成目标元素包络区域；及
区域融合模块2210，用于融合元素区域和目标元素包络区域，获得目标元素区域轮廓。
本实施例中，由目标元素区域检测模块通过基于人工智能的目标元素区域检测方式检测出待处理图像中的目标元素区域，并由包络生成模块对标元素区域寻找包络得到目标元素包络区域，再由区域融合模块融合由元素区域检测模块通过元素区域检测确定的元素区域和目标元素包络区域，得到目标元素区域轮廓。通过对基于人工智能的目标元素区域检测方式检测出目标元素区域寻找包络，可以有效利用人工智能的目标元素区域检测方式的检测结果，得到准确的目标元素包络区域，同时通过融合待处理图像中的元素区域和目标元素包络区域，可以利用元素区域检测的结果对目标元素包络区域进行校正，提高了检测划分的目标元素区 域轮廓的准确度。
在一个实施例中，元素区域检测模块2204包括通道差值特征确定模块、色彩分割模块和二值化分割模块；其中：通道差值特征确定模块，用于确定待处理图像中各像素点分别对应的通道差值特征；色彩分割模块，用于基于各像素点分别对应的通道差值特征将待处理图像进行色彩分割，获得色彩分割图像；二值化分割模块，用于对色彩分割图像进行二值化分割，获得待处理图像中的元素区域。
在一个实施例中，通道差值特征确定模块包括像素值确定模块、通道值确定模块和最大通道差确定模块；其中：像素值确定模块，用于确定待处理图像中各像素点分别对应的像素值；通道值确定模块，用于根据像素值得到相应像素点的各色彩通道值；最大通道差确定模块，用于确定各色彩通道值之间的最大通道差值，并根据最大通道差值得到通道差值特征；色彩分割模块，还用于按照各像素点对应的最大通道差值对待处理图像进行像素值映射，获得色彩分割图像。
在一个实施例中，二值化分割模块包括去噪模块、分割阈值获取模块和分割处理模块；其中：去噪模块，用于对色彩分割图像进行去噪处理，得到去噪图像；分割阈值获取模块，用于获取二值化分割阈值；分割处理模块，用于基于二值化分割阈值和去噪图像中各像素点对应的像素值，对去噪图像进行二值化分割，得到待处理图像中的元素区域。
在一个实施例中，分割处理模块包括像素值分析模块、二值化映射模块和映射结果分析模块；其中：像素值分析模块，用于确定去噪图像中各像素点对应的像素值；二值化映射模块，用于基于二值化分割阈值分别对去噪图像中各像素点对应的像素值进行像素二值化映射，获得去噪图像中各像素点对应的像素映射结果；映射结果分析模块，用于根据去噪图像中各像素点对应的像素映射结果获得待处理图像中的元素区域。
在一个实施例中，目标元素区域检测模块2206包括模型查询模块、模型处理模块和模型结果处理模块；其中：模型查询模块，用于查询采用标注目标元素区域的训练图像数据预训练的目标元素区域检测模型；模型处理模块，用于通过目标元素区域检测模型对待处理图像进行目标元素区域检测，得到目标元素区域概率图；模型结果处理模块，用于对目标元素区域概率图进行目标元素区域判定，根据判定结果得到待处理图像中的目标元素区域。
在一个实施例中，模型处理模块包括感兴趣区域模块、概率图获得模块和概率图拼接模块；其中：感兴趣区域模块，用于获取待处理图像经过区域划分处理获得的各感兴趣区域；概率图获得模块，用于分别将各感兴趣区域输入目标元素区域检测模型进行目标元素区域检测，得到目标元素区域检测模型输出的各感兴趣区域分别对应的感兴趣区域概率图；概率图拼接模块，用于将各感兴趣区域概率图进行拼接，得到待处理图像对应的目标元素区域概率图。
在一个实施例中，模型结果处理模块包括概率阈值获取模块、概率阈值映射模块和目标元素区域模块；其中：概率阈值获取模块，用于获取概率阈值；概率阈值映射模块，用于基于概率阈值和目标元素区域概率图中各像素点对应的概率值，对目标元素区域概率图进行概率二值化映射，获得目标元素区域概率图中各像素点的概率映射结果；目标元素区域模块，用于根据目标元素区域概率图中各像素点的概率映射结果得到待处理图像中的目标元素区域。
在一个实施例中，包络生成模块2208包括区域连通模块、区域填充模块和区域扩张模块；其中：区域连通模块，用于对待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域；区域填充模块，用于去除处于目标元素连通区域内的非目标元素区域，得到更新后的目标元素连通区域；区域扩张模块，用于对更新后的目标元素连通区域进行形态学扩张，获得目标元素包络区域。
在一个实施例中，区域连通模块包括滤波区域获得模块、区域类型分析模块和连通处理模块；其中：滤波区域获得模块，用于将待处理图像中的目标元素区域和非目标区域进行滤波划分，得到各滤波区域；区域类型分析模块，用于基于各滤波区域中像素点的像素值进行 区域类型分析，获得分析结果；连通处理模块，用于将分析结果为连通区域类型的滤波区域进行连通，生成目标元素连通区域。
在一个实施例中，还包括叠加处理模块，用于将目标元素区域轮廓叠加至待处理图像中，并通过预定的标识方式在待处理图像中标识出目标元素区域轮廓。
关于基于人工智能的图像处理装置的具体限定可以参见上文中对于基于人工智能的图像处理方法的限定。上述基于人工智能的图像处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图23所示。该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种基于人工智能的图像处理方法。
本领域技术人员可以理解，图23中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，还提供了一种计算机设备，包括存储器和一个或多个处理器，存储器中存储有计算机可读指令，该计算机可读指令被处理器执行时，使得一个或多个处理器执行上述各方法实施例中的步骤。
在一个实施例中，提供了一个或多个存储有计算机可读指令的非易失性可读存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述各方法实施例中的步骤。
在一个实施例中，提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机可读指令，该计算机可读指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机可读指令，处理器执行该计算机可读指令，使得该计算机设备执行上述各方法实施例中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory，DRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (21)

  1. 一种基于人工智能的图像处理方法，由计算机设备执行，其特征在于，所述方法包括：
    获取待处理图像；
    对所述待处理图像进行元素区域检测，以确定所述待处理图像中的元素区域；
    采用基于人工智能的目标元素区域检测方式，检测出所述待处理图像中的目标元素区域；
    对所检测出的所述目标元素区域寻找包络，生成目标元素包络区域；及
    融合所述元素区域和所述目标元素包络区域，获得目标元素区域轮廓。
  2. 根据权利要求1所述的方法，所述对所述待处理图像进行元素区域检测，以确定所述待处理图像中的元素区域，包括：
    确定所述待处理图像中各像素点分别对应的通道差值特征；
    基于所述各像素点分别对应的通道差值特征将所述待处理图像进行色彩分割，获得色彩分割图像；及
    对所述色彩分割图像进行二值化分割，获得所述待处理图像中的元素区域。
  3. 根据权利要求2所述的方法，所述确定所述待处理图像中各像素点分别对应的通道差值特征，包括：
    确定所述待处理图像中各像素点分别对应的像素值；
    根据所述像素值得到相应像素点的各色彩通道值；及
    确定所述各色彩通道值之间的最大通道差值，并根据所述最大通道差值得到通道差值特征；
    所述基于所述各像素点分别对应的通道差值特征将所述待处理图像进行色彩分割，获得色彩分割图像，包括：
    按照所述各像素点对应的最大通道差值对所述待处理图像进行像素值映射，获得色彩分割图像。
  4. 根据权利要求2所述的方法，所述对所述色彩分割图像进行二值化分割，获得所述待处理图像中的元素区域，包括：
    对所述色彩分割图像进行去噪处理，得到去噪图像；
    获取二值化分割阈值；及
    基于所述二值化分割阈值和所述去噪图像中各像素点对应的像素值，对所述去噪图像进行二值化分割，得到所述待处理图像中的元素区域。
  5. 根据权利要求4所述的方法，所述基于所述二值化分割阈值和所述去噪图像中各像素点对应的像素值，对所述去噪图像进行二值化分割，得到所述待处理图像中的元素区域，包括：
    确定所述去噪图像中各像素点对应的像素值；
    基于所述二值化分割阈值分别对所述去噪图像中各像素点对应的像素值进行像素二值化映射，获得所述去噪图像中各像素点对应的像素映射结果；及
    根据所述去噪图像中各像素点对应的像素映射结果获得所述待处理图像中的元素区域。
  6. 根据权利要求1所述的方法，所述采用基于人工智能的目标元素区域检测方式，检测出所述待处理图像中的目标元素区域，包括：
    查询采用标注目标元素区域的训练图像数据预训练的目标元素区域检测模型；
    通过所述目标元素区域检测模型对所述待处理图像进行目标元素区域检测，得到目标元素区域概率图；及
    对所述目标元素区域概率图进行目标元素区域判定，根据判定结果得到所述待处理图像中的目标元素区域。
  7. 根据权利要求6所述的方法，所述通过所述目标元素区域检测模型对所述待处理图像进行目标元素区域检测，得到目标元素区域概率图，包括：
    获取所述待处理图像经过区域划分处理获得的各感兴趣区域；
    分别将所述各感兴趣区域输入所述目标元素区域检测模型进行目标元素区域检测，得到所述目标元素区域检测模型输出的所述各感兴趣区域分别对应的感兴趣区域概率图；及
    将各所述感兴趣区域概率图进行拼接，得到所述待处理图像对应的目标元素区域概率图。
  8. 根据权利要求6所述的方法，所述对所述目标元素区域概率图进行目标元素区域判定，根据判定结果得到所述待处理图像中的目标元素区域，包括：
    获取概率阈值；
    基于所述概率阈值和所述目标元素区域概率图中各像素点对应的概率值，对所述目标元素区域概率图进行概率二值化映射，获得所述目标元素区域概率图中各像素点的概率映射结果；及
    根据所述目标元素区域概率图中各像素点的概率映射结果得到所述待处理图像中的目标元素区域。
  9. 根据权利要求1至8任意一项所述的方法，所述对所检测出的所述目标元素区域寻找包络，生成目标元素包络区域，包括：
    对所述待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域；
    去除处于所述目标元素连通区域内的非目标元素区域，得到更新后的目标元素连通区域；及
    对更新后的目标元素连通区域进行形态学扩张，获得目标元素包络区域。
  10. 根据权利要求9所述的方法，所述对所述待处理图像中的目标元素区域进行区域连通，生成目标元素连通区域，包括：
    将所述待处理图像中的目标元素区域和非目标区域进行滤波划分，得到各滤波区域；
    基于各所述滤波区域中像素点的像素值进行区域类型分析，获得分析结果；及
    将分析结果为连通区域类型的滤波区域进行连通，生成目标元素连通区域。
  11. 根据权利要求1至10任意一项所述的方法，所述方法还包括：
    将所述目标元素区域轮廓叠加至所述待处理图像中，并通过预定的标识方式在所述待处理图像中标识出所述目标元素区域轮廓。
  12. 一种基于人工智能的图像处理装置，其特征在于，所述装置包括：
    图像获取模块，用于获取待处理图像；
    元素区域检测模块，用于对所述待处理图像进行元素区域检测，以确定所述待处理图像中的元素区域；
    目标元素区域检测模块，用于采用基于人工智能的目标元素区域检测方式，检测出所述待处理图像中的目标元素区域；
    包络生成模块，用于对所检测出的所述目标元素区域寻找包络，生成目标元素包络区域；及
    区域融合模块，用于融合所述元素区域和所述目标元素包络区域，获得目标元素区域轮廓。
  13. 根据权利要求12所述的装置，所述元素区域检测模块包括：
    通道差值特征确定模块，用于确定所述待处理图像中各像素点分别对应的通道差值特征；
    色彩分割模块，用于基于所述各像素点分别对应的通道差值特征将所述待处理图像进行色彩分割，获得色彩分割图像；及
    二值化分割模块，用于对所述色彩分割图像进行二值化分割，获得所述待处理图像中的元素区域。
  14. 根据权利要求13所述的装置，所述通道差值特征确定模块包括：
    像素值确定模块，用于确定所述待处理图像中各像素点分别对应的像素值；
    通道值确定模块，用于根据所述像素值得到相应像素点的各色彩通道值；及
    最大通道差确定模块，用于确定所述各色彩通道值之间的最大通道差值，并根据所述最大通道差值得到通道差值特征；
    所述色彩分割模块，还用于按照所述各像素点对应的最大通道差值对所述待处理图像进行像素值映射，获得色彩分割图像。
  15. 根据权利要求13所述的装置，所述二值化分割模块包括：
    去噪模块，用于对所述色彩分割图像进行去噪处理，得到去噪图像；
    分割阈值获取模块，用于获取二值化分割阈值；及
    分割处理模块，用于基于所述二值化分割阈值和所述去噪图像中各像素点对应的像素值，对所述去噪图像进行二值化分割，得到所述待处理图像中的元素区域。
  16. 根据权利要求15所述的装置，所述分割处理模块包括：
    像素值分析模块，用于确定所述去噪图像中各像素点对应的像素值；
    二值化映射模块，用于基于所述二值化分割阈值分别对所述去噪图像中各像素点对应的像素值进行像素二值化映射，获得所述去噪图像中各像素点对应的像素映射结果；及
    映射结果分析模块，用于根据所述去噪图像中各像素点对应的像素映射结果获得所述待处理图像中的元素区域。
  17. 根据权利要求12所述的装置，所述目标元素区域检测模块包括：
    模型查询模块，用于查询采用标注目标元素区域的训练图像数据预训练的目标元素区域检测模型；
    模型处理模块，用于通过所述目标元素区域检测模型对所述待处理图像进行目标元素区域检测，得到目标元素区域概率图；及
    模型结果处理模块，用于对所述目标元素区域概率图进行目标元素区域判定，根据判定结果得到所述待处理图像中的目标元素区域。
  18. 根据权利要求17所述的装置，所述模型处理模块包括：
    感兴趣区域模块，用于获取所述待处理图像经过区域划分处理获得的各感兴趣区域；
    概率图获得模块，用于分别将所述各感兴趣区域输入所述目标元素区域检测模型进行目标元素区域检测，得到所述目标元素区域检测模型输出的所述各感兴趣区域分别对应的感兴趣区域概率图；及
    概率图拼接模块，用于将各所述感兴趣区域概率图进行拼接，得到所述待处理图像对应的目标元素区域概率图。
  19. 一种计算机设备，包括存储器和一个或多个处理器，所述存储器存储有计算机可读指令，其特征在于，所述计算机可读指令被所述处理器执行时，使得所述一个或多个处理器执行权利要求1至11中任一项所述的方法的步骤。
  20. 一个或多个存储有计算机可读指令的非易失性可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行权利要求1至11中任一项所述的方法的步骤。
  21. 一种计算机程序产品，包括计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至11任一项所述的方法的步骤。
PCT/CN2021/124384 2020-11-02 2021-10-18 基于人工智能的图像处理方法、装置、计算机设备和存储介质 WO2022089236A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21884964.4A EP4198889A4 (en) 2020-11-02 2021-10-18 IMAGE PROCESSING METHOD AND APPARATUS BASED ON ARTIFICIAL INTELLIGENCE, AND COMPUTER DEVICE AND STORAGE MEDIUM
JP2023513664A JP7412636B2 (ja) 2020-11-02 2021-10-18 人工知能に基づく画像処理方法、装置、コンピュータ装置及びコンピュータプログラム
US17/960,804 US20230023585A1 (en) 2020-11-02 2022-10-05 Artificial intelligence-based image processing method and apparatus, computer device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011202281.0A CN112330688A (zh) 2020-11-02 2020-11-02 基于人工智能的图像处理方法、装置和计算机设备
CN202011202281.0 2020-11-02

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/960,804 Continuation US20230023585A1 (en) 2020-11-02 2022-10-05 Artificial intelligence-based image processing method and apparatus, computer device and storage medium

Publications (1)

Publication Number Publication Date
WO2022089236A1 true WO2022089236A1 (zh) 2022-05-05

Family

ID=74324041

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/124384 WO2022089236A1 (zh) 2020-11-02 2021-10-18 基于人工智能的图像处理方法、装置、计算机设备和存储介质

Country Status (5)

Country Link
US (1) US20230023585A1 (zh)
EP (1) EP4198889A4 (zh)
JP (1) JP7412636B2 (zh)
CN (1) CN112330688A (zh)
WO (1) WO2022089236A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115170974A (zh) * 2022-09-08 2022-10-11 深圳市勘察研究院有限公司 基于ai智能检测溶洞连通性的方法及装置
CN115797494A (zh) * 2023-02-06 2023-03-14 武汉精臣智慧标识科技有限公司 图片二值化处理方法、装置、电子设备及存储介质
CN116342592A (zh) * 2023-05-25 2023-06-27 山东沐泽中药饮片有限公司 用于蟾酥加工的蟾蜍腺体浆液烘干监测方法
CN116664550A (zh) * 2023-07-10 2023-08-29 广州医科大学附属第一医院（广州呼吸中心） 肺癌组织免疫组化pd-l1病理切片的智能识别方法及装置
CN117197006A (zh) * 2023-09-27 2023-12-08 上海世禹精密设备股份有限公司 区域过滤方法、装置、电子设备及计算机可读存储介质
CN117422716A (zh) * 2023-12-19 2024-01-19 沂水友邦养殖服务有限公司 基于人工智能的肉鸡养殖生态预警方法及系统
CN117474902A (zh) * 2023-12-25 2024-01-30 山东明佳科技有限公司 一种桶面料包的缺包检测方法、系统、设备和存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109934812B (zh) * 2019-03-08 2022-12-09 腾讯科技（深圳）有限公司 图像处理方法、装置、服务器及存储介质
CN112330688A (zh) * 2020-11-02 2021-02-05 腾讯科技（深圳）有限公司 基于人工智能的图像处理方法、装置和计算机设备
CN112950700B (zh) * 2021-02-25 2022-10-04 安徽农业大学 基于深度学习和图像处理的植物叶片气孔导度测量方法
CN113744126A (zh) * 2021-08-06 2021-12-03 Oppo广东移动通信有限公司 图像处理方法及装置、计算机可读介质和电子设备
CN115690130B (zh) * 2022-12-30 2023-06-27 杭州咏柳科技有限公司 一种图像处理方法和装置
CN116152248B (zh) * 2023-04-20 2023-06-30 中科慧远视觉技术(北京)有限公司 外观缺陷的检测方法及装置、存储介质、计算机设备
CN116503804B (zh) * 2023-04-28 2023-11-10 东莞理工学院 基于图像处理的粗集料形态特征获取方法和系统
CN116630325B (zh) * 2023-07-25 2023-10-13 苏州赫芯科技有限公司 基于电路板的uv胶检测方法、设备和存储介质
CN117173185B (zh) * 2023-11-03 2024-01-19 东北大学 轧制板材的区域检测方法及装置、存储介质、计算机设备
CN117333586A (zh) * 2023-11-30 2024-01-02 浙江口碑网络技术有限公司 一种图像处理方法及装置、存储介质和电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9779504B1 (en) * 2011-12-14 2017-10-03 Atti International Services Company, Inc. Method and system for identifying anomalies in medical images especially those including one of a pair of symmetric body parts
CN110059685A (zh) * 2019-04-26 2019-07-26 腾讯科技（深圳）有限公司 文字区域检测方法、装置及存储介质
CN110118775A (zh) * 2019-05-10 2019-08-13 重庆交通大学 厂拌水泥稳定碎石骨料组成快速检测方法
WO2019235298A1 (ja) * 2018-06-07 2019-12-12 富士フイルム株式会社 画像診断支援装置、画像診断支援方法、及び画像診断支援プログラム
CN111047572A (zh) * 2019-12-10 2020-04-21 南京安科医疗科技有限公司 一种基于Mask RCNN的医学图像中脊柱自动定位方法
CN112330688A (zh) * 2020-11-02 2021-02-05 腾讯科技（深圳）有限公司 基于人工智能的图像处理方法、装置和计算机设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003520658A (ja) 2000-01-27 2003-07-08 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 脊柱の幾何学的データを抽出する方法及びシステム
WO2012016242A2 (en) * 2010-07-30 2012-02-02 Aureon Biosciences, Inc. Systems and methods for segmentation and processing of tissue images and feature extraction from same for treating, diagnosing, or predicting medical conditions
CN108898610B (zh) * 2018-07-20 2020-11-20 电子科技大学 一种基于mask-RCNN的物体轮廓提取方法
CN110909756A (zh) * 2018-09-18 2020-03-24 苏宁 用于医学图像识别的卷积神经网络模型训练方法和装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9779504B1 (en) * 2011-12-14 2017-10-03 Atti International Services Company, Inc. Method and system for identifying anomalies in medical images especially those including one of a pair of symmetric body parts
WO2019235298A1 (ja) * 2018-06-07 2019-12-12 富士フイルム株式会社 画像診断支援装置、画像診断支援方法、及び画像診断支援プログラム
CN110059685A (zh) * 2019-04-26 2019-07-26 腾讯科技（深圳）有限公司 文字区域检测方法、装置及存储介质
CN110118775A (zh) * 2019-05-10 2019-08-13 重庆交通大学 厂拌水泥稳定碎石骨料组成快速检测方法
CN111047572A (zh) * 2019-12-10 2020-04-21 南京安科医疗科技有限公司 一种基于Mask RCNN的医学图像中脊柱自动定位方法
CN112330688A (zh) * 2020-11-02 2021-02-05 腾讯科技（深圳）有限公司 基于人工智能的图像处理方法、装置和计算机设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4198889A4

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115170974A (zh) * 2022-09-08 2022-10-11 深圳市勘察研究院有限公司 基于ai智能检测溶洞连通性的方法及装置
CN115170974B (zh) * 2022-09-08 2022-12-20 深圳市勘察研究院有限公司 基于ai智能检测溶洞连通性的方法及装置
CN115797494A (zh) * 2023-02-06 2023-03-14 武汉精臣智慧标识科技有限公司 图片二值化处理方法、装置、电子设备及存储介质
CN115797494B (zh) * 2023-02-06 2023-05-23 武汉精臣智慧标识科技有限公司 图片二值化处理方法、装置、电子设备及存储介质
CN116342592A (zh) * 2023-05-25 2023-06-27 山东沐泽中药饮片有限公司 用于蟾酥加工的蟾蜍腺体浆液烘干监测方法
CN116342592B (zh) * 2023-05-25 2023-07-25 山东沐泽中药饮片有限公司 用于蟾酥加工的蟾蜍腺体浆液烘干监测方法
CN116664550A (zh) * 2023-07-10 2023-08-29 广州医科大学附属第一医院（广州呼吸中心） 肺癌组织免疫组化pd-l1病理切片的智能识别方法及装置
CN116664550B (zh) * 2023-07-10 2024-04-12 广州医科大学附属第一医院（广州呼吸中心） 肺癌组织免疫组化pd-l1病理切片的智能识别方法及装置
CN117197006A (zh) * 2023-09-27 2023-12-08 上海世禹精密设备股份有限公司 区域过滤方法、装置、电子设备及计算机可读存储介质
CN117197006B (zh) * 2023-09-27 2024-05-10 上海世禹精密设备股份有限公司 区域过滤方法、装置、电子设备及计算机可读存储介质
CN117422716A (zh) * 2023-12-19 2024-01-19 沂水友邦养殖服务有限公司 基于人工智能的肉鸡养殖生态预警方法及系统
CN117422716B (zh) * 2023-12-19 2024-03-08 沂水友邦养殖服务有限公司 基于人工智能的肉鸡养殖生态预警方法及系统
CN117474902A (zh) * 2023-12-25 2024-01-30 山东明佳科技有限公司 一种桶面料包的缺包检测方法、系统、设备和存储介质
CN117474902B (zh) * 2023-12-25 2024-03-12 山东明佳科技有限公司 一种桶面料包的缺包检测方法、系统、设备和存储介质

Also Published As

Publication number Publication date
US20230023585A1 (en) 2023-01-26
EP4198889A1 (en) 2023-06-21
CN112330688A (zh) 2021-02-05
EP4198889A4 (en) 2024-02-07
JP2023544967A (ja) 2023-10-26
JP7412636B2 (ja) 2024-01-12

Similar Documents

Publication Publication Date Title
WO2022089236A1 (zh) 基于人工智能的图像处理方法、装置、计算机设备和存储介质
CN108010021B (zh) 一种医学图像处理系统和方法
WO2020224424A1 (zh) 图像处理方法、装置、计算机可读存储介质和计算机设备
CN109523526B (zh) 组织结节检测及其模型训练方法、装置、设备和系统
CN111127466B (zh) 医学图像检测方法、装置、设备及存储介质
Oskal et al. A U-net based approach to epidermal tissue segmentation in whole slide histopathological images
Jerripothula et al. Cats: Co-saliency activated tracklet selection for video co-localization
Pan et al. Cell detection in pathology and microscopy images with multi-scale fully convolutional neural networks
Gadermayr et al. CNN cascades for segmenting whole slide images of the kidney
CN112419295A (zh) 医学图像处理方法、装置、计算机设备和存储介质
US11176412B2 (en) Systems and methods for encoding image features of high-resolution digital images of biological specimens
Sánchez et al. Bright spot regions segmentation and classification for specular highlights detection in colonoscopy videos
Zhao et al. Automatic blur region segmentation approach using image matting
CN115063592B (zh) 一种基于多尺度的全扫描病理特征融合提取方法及系统
Rempfler et al. Tracing cell lineages in videos of lens-free microscopy
EP3440629B1 (en) Spatial index creation for ihc image analysis
Wang et al. Automatic generation of pathological benchmark dataset from hyperspectral images of double stained tissues
Hasan et al. Automated masks generation for coffee and apple leaf infected with single or multiple diseases-based color analysis approaches
Khan et al. Segmentation of single and overlapping leaves by extracting appropriate contours
Wang et al. Semantic feature based multi-spectral saliency detection
Rehman et al. Machine learning based skin lesion segmentation method with novel borders and hair removal techniques
Park et al. Segmentation-based tracking of macrophages in 2D+ time microscopy movies inside a living animal
Li et al. Inflammatory cells detection in H&E staining histology images using deep convolutional neural network with distance transformation
Khadangi et al. Automated framework to reconstruct 3D model of cardiac Z-disk: an image processing approach
Khryashchev et al. Histological Images Segmentation by Convolutional Neural Network with Morphological Post-filtration

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21884964

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023513664

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021884964

Country of ref document: EP

Effective date: 20230314

NENP Non-entry into the national phase

Ref country code: DE