WO2020052668A1 - 图像处理方法、电子设备及存储介质 - Google Patents

图像处理方法、电子设备及存储介质 Download PDF

Info

Publication number
WO2020052668A1
WO2020052668A1 PCT/CN2019/105787 CN2019105787W WO2020052668A1 WO 2020052668 A1 WO2020052668 A1 WO 2020052668A1 CN 2019105787 W CN2019105787 W CN 2019105787W WO 2020052668 A1 WO2020052668 A1 WO 2020052668A1
Authority
WO
WIPO (PCT)
Prior art keywords
instance
image
center
pixel
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/105787
Other languages
English (en)
French (fr)
Inventor
李嘉辉
胡志强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201811077349.XA external-priority patent/CN109389129B/zh
Priority claimed from CN201811077358.9A external-priority patent/CN109345540B/zh
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to SG11202013059VA priority Critical patent/SG11202013059VA/en
Priority to JP2020564310A priority patent/JP7026826B2/ja
Publication of WO2020052668A1 publication Critical patent/WO2020052668A1/zh
Priority to US17/135,489 priority patent/US20210118144A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695—Preprocessing, e.g. image segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to an image processing method, an electronic device, and a storage medium.
  • Image processing also called image processing, is a technique that uses a computer to analyze an image to achieve the desired result.
  • Image processing generally refers to digital image processing.
  • a digital image refers to a two-dimensional array captured by an industrial camera, a video camera, a scanner, and other equipment.
  • the elements of the array are called pixels, and their values are called gray values.
  • Image processing plays a very important role in many fields, especially the processing of medical images.
  • Embodiments of the present disclosure provide an image processing method, an electronic device, and a storage medium.
  • a first aspect of an embodiment of the present disclosure provides an image processing method, including: processing a first image to obtain a prediction result of each of a plurality of pixels in the first image, where the prediction result includes a semantic prediction result and a center relative A position prediction result, wherein the semantic prediction result indicates that the pixel point is located in an instance area or a background area, and the center relative position prediction result indicates a relative position between the pixel point and the instance center; based on the plurality of pixels A semantic prediction result and a center relative position prediction result of each pixel in the point determine an instance segmentation result of the first image.
  • processing the first image to obtain a semantic prediction result of multiple pixels in the first image includes processing the first image to obtain multiple pixels in the first image.
  • the predicted probability of the instance area, the predicted probability of the instance area indicates the probability that the pixel is located in the instance area; based on the second threshold, the predicted probability of the instance area of the multiple pixels is binarized to obtain the multiple pixels Semantic prediction results for each pixel in.
  • the instance center region includes a region within the instance region and smaller than the instance region, and a geometric center of the instance center region and a geometric center of the instance region overlap.
  • the method before processing the first image, further includes: preprocessing the second image to obtain the first image, so that the first image meets a preset Contrast and / or preset gray value.
  • the method before processing the first image, further includes: preprocessing the second image to obtain the first image, so that the first image The image meets the preset image size.
  • the determining an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each pixel in the multiple pixels includes: The semantic prediction result of each pixel point in the plurality of pixel points is determined from the plurality of pixel points, and at least one first pixel point located in the instance area is determined from the plurality of pixel points; for each of the first pixel points, based on the first The center relative position prediction result of the pixel point determines the instance to which the first pixel point belongs.
  • the example is a segmentation object in the first image, and may specifically be a closed structure in the first image.
  • Examples in the embodiments of the present disclosure include nuclei, that is, the embodiments of the present disclosure can be applied to cell division.
  • the prediction result further includes a center area prediction result, where the center area prediction result indicates whether the pixel point is located in an instance center area.
  • the method further includes: determining at least one instance central region of the first image based on a prediction result of a central region of each of the plurality of pixel points; and based on the first pixel Determining a center relative position prediction result of the point, and determining an instance to which the first pixel point belongs, includes: determining the first pixel from a center area of the at least one instance based on the center relative position prediction result of the first pixel point Point to the instance center area.
  • the determining a center area of at least one instance of the first image based on a prediction result of a center area of each of the plurality of pixel points includes: The prediction result of the central area of each pixel in the pixel points is subjected to a connected domain search process on the first image to obtain at least one instance central area.
  • the performing a connected domain search process on the first image based on a prediction result of a central area of each of the plurality of pixel points to obtain at least one instance central area includes: Based on the prediction result of the central area of each of the plurality of pixel points, a connected domain search process is performed on the first image using a random walk algorithm to obtain at least one instance central area.
  • determining the instance center area corresponding to the first pixel point from the at least one instance center area based on the center relative position prediction result of the first pixel point includes: The position information of the first pixel point and a center relative position prediction result of the first pixel point, determining a center prediction position of the first pixel point; based on the center prediction position of the first pixel point and the at least one The location information of the instance central area determines the instance central area corresponding to the first pixel point from the at least one instance central area.
  • the first pixel is determined from the at least one instance center region based on a center prediction position of the first pixel point and position information of the at least one instance center region.
  • the instance center area corresponding to the point includes: in response to the center predicted position of the first pixel point belonging to a first instance center area in the at least one instance center area, determining the first instance center area as the first An instance center area corresponding to one pixel point; or, in response to the center predicted position of the first pixel point not belonging to any instance center area in the at least one instance center area, The instance center area closest to the center prediction position of the first pixel point is determined as the instance center area corresponding to the first pixel point.
  • the processing the first image to obtain a prediction result of multiple pixels in the first image includes processing the first image to obtain the first image. Prediction probability of the central region of multiple pixels in the image; performing a binarization process on the predicted probability of the central region of the plurality of pixels based on a first threshold to obtain a prediction of the central region of each of the plurality of pixels result.
  • the processing the first image to obtain a prediction result of multiple pixels in the first image includes: inputting the first image to a neural network for processing, and outputting the first image. Prediction results of multiple pixels in the first image.
  • a second aspect of the embodiments of the present disclosure provides an electronic device including a prediction module and a segmentation module, wherein the prediction module is configured to process a first image to obtain a prediction result of multiple pixels in the first image.
  • the prediction result includes a semantic prediction result and a center relative position prediction result, wherein the semantic prediction result indicates that the pixel point is located in an instance area or a background area, and the center relative position prediction result indicates that the pixel point and the instance center A relative position between the two;
  • the segmentation module configured to determine an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each of the plurality of pixel points.
  • the prediction module is specifically configured to process the first image to obtain an instance area prediction probability of multiple pixels in the first image, where the instance area prediction probability indicates that the pixel is located in an instance The probability of the region; based on the second threshold, binarizing the prediction probability of the above-mentioned example regions of the plurality of pixels to obtain a semantic prediction result of each of the plurality of pixels.
  • the electronic device further includes a pre-processing module for pre-processing the second image to obtain the first image, so that the first image satisfies a preset contrast and / Or preset gray value.
  • the pre-processing module is further configured to pre-process the second image to obtain the first image, so that the first image meets a preset image size.
  • the segmentation module includes a first unit and a second unit, wherein: the first unit is configured to: based on a semantic prediction result of each pixel in the plurality of pixels, from Determining, among the plurality of pixels, at least one first pixel located in an instance area; the second unit is configured to determine, based on a prediction result of a center relative position of each first pixel in the at least one first pixel, The instance to which each first pixel point belongs.
  • the prediction result further includes a center area prediction result, where the center area prediction result indicates whether the pixel point is located in an instance center area
  • the segmentation module further includes a third unit for: Determining at least one instance central area of the first image based on a prediction result of a central area of each of the plurality of pixel points; the second unit is specifically configured to, based on the at least one first pixel point, The prediction result of the center relative position of each first pixel point determines the instance center area corresponding to each first pixel point from the at least one instance center area.
  • the third unit is specifically configured to perform a connected domain search process on the first image based on a prediction result of a central area of each pixel of the multiple pixels to obtain Central area of at least one instance.
  • the third unit is specifically configured to use a random walk algorithm to connect the first image based on a prediction result of a central area of each pixel in the plurality of pixels. Domain search processing to obtain at least one instance central area.
  • the second unit is specifically configured to determine the first pixel point based on the position information of the first pixel point and a center relative position prediction result of the first pixel point. Determine the center location of the instance corresponding to the first pixel point from the at least one instance center area based on the center prediction location of the first pixel point and the position information of the at least one instance center area.
  • the second unit is specifically configured to: in response to a center predicted position of the first pixel point belonging to a first instance center region among the at least one instance center region, The first instance central area is determined as the instance central area corresponding to the first pixel point.
  • the second unit is specifically configured to: in response to that the center predicted position of the first pixel point does not belong to any instance center region of the at least one instance center region, The instance center area that is closest to the center prediction position of the first pixel point in the at least one instance center area is determined as the instance center area corresponding to the first pixel point.
  • the prediction module includes a probability prediction unit and a judgment unit, wherein the probability prediction unit is configured to process the first image to obtain a plurality of the first image.
  • the predicted probability of the central region of the pixel; the determining unit is configured to perform a binarization process on the predicted probability of the central region of the plurality of pixels based on a first threshold to obtain a Center area prediction results.
  • the prediction module is specifically configured to input a first image to a neural network for processing, and output prediction results of multiple pixels in the first image.
  • the instance segmentation result of the first image is determined based on the semantic prediction result and the center relative position prediction result of each pixel point among the multiple pixel points included in the first image, so that the instance in image processing can be obtained. Segmentation has the advantages of fast speed and high accuracy.
  • a third aspect of the embodiments of the present disclosure provides an image processing method, including: obtaining N sets of instance segmentation output data, where the N sets of instance segmentation output data are instance segmentation outputs obtained by processing images by N instance segmentation models, respectively.
  • the segmented output data of the N groups of instances have different data structures, where N is an integer greater than 1.
  • segmenting the output data based on the N sets of instances to obtain integrated semantic data and integrated central area data of the image, wherein, the integrated semantic data indicates the pixels located in the instance area in the image, and the integrated central area data indicates the pixels located in the instance center area in the image; the integrated semantic data and the integrated central area based on the image Data to obtain instance segmentation results of the image.
  • the segmenting output data based on the N groups of instances to obtain the integrated semantic data and integration center area data of the image includes: segmenting each instance in the model for the N instances Segmentation model, based on the instance segmentation output data of the instance segmentation model, to obtain the semantic data and central area data of the instance segmentation model; based on the semantic data and central area data of each instance segmentation model in the N instance segmentation models To obtain integrated semantic data and integrated central area data of the image.
  • the instance segmentation output data based on the instance segmentation model to obtain semantic data and central area data of the instance segmentation model includes: instance segmentation output based on the instance segmentation model.
  • Data determining instance identification information corresponding to each pixel in multiple pixels of the image in the instance segmentation model; based on the instance corresponding to each pixel in the multiple pixels in the instance segmentation model Identifying information to obtain a semantic prediction value of each pixel in the instance segmentation model, wherein the semantic data of the instance segmentation model includes a semantic prediction value of each pixel among a plurality of pixels of the image .
  • the instance segmentation output data based on the instance segmentation model to obtain semantic data and central area data of the instance segmentation model further includes: instance segmentation based on the instance segmentation model.
  • the instance segmentation output data based on the instance segmentation model determining that in the instance segmentation model, the image is located before at least two pixels of the instance area, and further includes: Erosion processing is performed on the instance segmentation output data of the instance segmentation model to obtain the erosion data of the instance segmentation model.
  • the instance segmentation output data based on the instance segmentation model, and determining, in the instance segmentation model, at least two pixels in the image located in an instance region include: based on the instance segmentation model The corrosion data is determined, in the instance segmentation model, at least two pixels in the image located in the instance area.
  • the determining the instance center position of the instance segmentation model based on the position information of at least two pixels located in the instance region in the instance segmentation model includes: The average value of the positions of at least two pixels of the region is used as the instance center position of the instance segmentation model.
  • determining the instance center area of the instance segmentation model based on the instance center position of the instance segmentation model and the position information of the at least two pixels includes: The instance center position of the instance segmentation model and the position information of the at least two pixels determine the maximum distance between the at least two pixels and the instance center position; based on the maximum distance, determine a first threshold; The pixel point having a distance between the at least two pixel points and the center position of the instance that is less than or equal to the first threshold is determined as a pixel point in the center area of the instance.
  • the obtaining the integrated semantic data and the integrated central area data of the image based on the semantic data and the central area data of each instance segmentation model in the N instance segmentation models includes: Based on the semantic data of each instance segmentation model in the N instance segmentation models, determine a semantic vote value of each pixel in the plurality of pixel points of the image; The semantic voting value is binarized to obtain the integrated semantic value of each pixel in the image, wherein the integrated semantic data of the image includes the integrated semantic value of each pixel in the plurality of pixels.
  • the binarizing the semantic voting value of each pixel in the multiple pixels to obtain the integrated semantic value of each pixel in the image includes: Based on the number N of the multiple instance segmentation models, a second threshold is determined; based on the second threshold, the semantic voting value of each pixel in the multiple pixels is binarized to obtain the The integrated semantic value of each pixel in the image.
  • the second threshold value is a round-up result of N / 2.
  • the obtaining an instance segmentation result of the image based on the integrated semantic data and integrated central area data of the image includes: obtaining the integrated central area data of the image based on the image. An at least one instance central region of the image; and based on the integrated semantic data of the at least one instance central region and the image, determining an instance to which each pixel of the plurality of pixels of the image belongs.
  • the determining, based on the integrated semantic data of the at least one instance central area and the image, an instance to which each pixel in a plurality of pixel points of the image belongs includes: An integrated semantic value of each pixel point in the plurality of pixel points of the image and a center region of the at least one instance are randomly walked to obtain an instance to which each pixel point belongs.
  • an electronic device including: an acquisition module, a conversion module, and a segmentation module, wherein the acquisition module is configured to acquire N sets of instance segmentation output data, wherein the N set of instance segmentation outputs The data is the instance segmentation output result obtained by processing the image by N instance segmentation models, and the N sets of instance segmentation output data have different data structures, where N is an integer greater than 1; the conversion module is used for Segment the output data based on the N sets of instances to obtain the integrated semantic data and integrated central area data of the image, where the integrated semantic data indicates pixels in the image that are located in the instance area, and the integrated central area data indicates Pixels in the image located in the central area of the instance; the segmentation module is configured to obtain an instance segmentation result of the image based on the integrated semantic data and integrated central area data of the image.
  • the conversion module includes a first conversion unit and a second conversion unit, wherein: the first conversion unit is configured to segment a model for each instance of the N instance segmentation models , Based on the instance segmentation output data of the instance segmentation model, to obtain semantic data and central area data of the instance segmentation model; the second conversion unit is configured to segment the model based on each instance of the N instance segmentation models To obtain the integrated semantic data and integrated central area data of the image.
  • the first conversion unit is specifically configured to: based on instance segmentation output data of the instance segmentation model, determine each of a plurality of pixels of the image in the instance segmentation model. Instance identification information corresponding to each pixel; based on the instance identification information corresponding to each pixel in the plurality of pixels in the instance segmentation model, obtaining a semantic prediction of each pixel in the instance segmentation model Value, wherein the semantic data of the instance segmentation model includes a semantic prediction value of each pixel in a plurality of pixels of the image.
  • the first conversion unit is further configured to: segment output data based on the instance segmentation model of the instance segmentation model, and determine that, in the instance segmentation model, the image is located in an instance region in the image. At least two pixels; determining an instance center position of the instance segmentation model based on position information of at least two pixels in the instance region in the instance segmentation model; based on the instance center position of the instance segmentation model and the instance segmentation model Position information of at least two pixels determines an instance central area of the instance segmentation model.
  • the conversion module further includes an corrosion processing unit, configured to perform corrosion processing on the instance segmentation output data of the instance segmentation model to obtain the erosion data of the instance segmentation model; the first conversion The unit is specifically configured to determine, based on the corrosion data of the instance segmentation model, at least two pixel points in the image that are located in the instance area.
  • the first conversion unit is specifically configured to use an average value of the positions of at least two pixels located in the instance area as an instance center position of the instance segmentation model.
  • the first conversion unit is further configured to determine the at least two pixels based on an instance center position of the instance segmentation model and position information of the at least two pixels.
  • the pixels are determined as the pixels in the central area of the instance.
  • the conversion module is specifically configured to: determine a semantic voting value of each pixel in a plurality of pixels of the image based on the semantic data of the instance segmentation model; The semantic voting value of each pixel in the plurality of pixels is binarized to obtain an integrated semantic value of each pixel in the image, wherein the integrated semantic data of the image includes the plurality of pixels Integrated semantic value of each pixel in.
  • the conversion module is further configured to: determine a second threshold value based on the number N of the multiple instance segmentation models; and based on the second threshold value, The semantic voting value of each pixel in the pixel is binarized to obtain the integrated semantic value of each pixel in the image.
  • the second threshold value is a round-up result of N / 2.
  • a fifth aspect of the embodiments of the present disclosure provides another electronic device, including a processor and a memory, where the memory is configured to store a computer program, the computer program is configured to be executed by the processor, and the processor is configured to execute Some or all of the steps described in the methods of the first aspect and the third aspect of the embodiments of the present disclosure.
  • a sixth aspect of the embodiments of the present disclosure provides a computer-readable storage medium for storing a computer program, wherein the computer program causes a computer to execute the first and third aspects of the embodiment of the present disclosure. Some or all of the steps described in either method.
  • the embodiment of the present disclosure is based on N sets of instance segmentation output data obtained by processing images through N instance segmentation models, to obtain integrated semantic data and integrated central area data of the above image, and then based on the integrated semantic data and integrated central area data of the above image.
  • FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure.
  • FIG. 3 is a schematic diagram of a segmentation result of a cell instance according to an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic flowchart of still another image processing method according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of an image representation form of cell instance segmentation according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of another electronic device according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of still another electronic device according to an embodiment of the present disclosure.
  • an embodiment herein means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present disclosure.
  • the appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they independent or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
  • the electronic device involved in the embodiment of the present disclosure may allow access by multiple other terminal devices.
  • the electronic device includes a terminal device.
  • the above-mentioned terminal devices include, but are not limited to, portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (eg, touch screen displays and / or touch pads). It should also be understood that, in some embodiments, the terminal device is not a portable communication device, but a desktop computer with a touch-sensitive surface (eg, a touch screen display and / or a touch pad).
  • Deep learning stems from the study of artificial neural networks. Multi-layer perceptron with multiple hidden layers is a deep learning structure. Deep learning combines low-level features to form more abstract high-level representation attribute categories or features to discover distributed feature representations of data.
  • Deep learning is a method based on representational learning of data in machine learning. Observed values (such as an image) can be represented in a variety of ways, such as a vector of intensity values for each pixel, or more abstractly represented as a series of edges, regions of a specific shape, and so on. It is easier to learn tasks from examples using some specific representation methods (for example, face recognition or facial expression recognition).
  • the benefit of deep learning is to replace unobtained features manually with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction. Deep learning is a new field in machine learning research. Its motivation is to build and simulate the neural network of the human brain for analysis and learning, so that it can mimic the mechanism of the human brain to interpret data, such as images, sounds, and text.
  • CNN Convolutional Neural Network
  • DBN Deep Belief Net
  • FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 1, the image processing method includes the following steps.
  • the first image is processed to obtain prediction results of multiple pixels in the first image.
  • the above prediction results include a semantic prediction result and a center relative position prediction result.
  • the semantic prediction result indicates that the pixel is located in the instance area or the background area
  • the center relative position prediction result indicates the relative position between the pixel and the instance center.
  • multiple pixels may be all or part of the pixels of the first image, which is not limited in the embodiment of the present disclosure.
  • the first image may include a pathological image obtained through various image acquisition devices (such as a microscope), such as a nuclear image.
  • image acquisition devices such as a microscope
  • the embodiment of the present disclosure does not limit the manner of obtaining the first image and the specific implementation of the example.
  • the first image may be processed in various ways. For example, an instance segmentation algorithm is used to process the first image, or the first image may be input to a neural network for processing and the prediction results of multiple pixels in the first image may be output. This embodiment of the present disclosure does not do this. limited.
  • a deep learning-based neural network may be used to obtain the prediction results of multiple pixels in the first image, such as a deep fusion network (Deepet Layer Aggregation, DLANet).
  • DLANet Deep fusion network
  • Deep fusion network also called deep aggregation network, expands the standard architecture through deeper aggregation to better integrate the information of each layer. Deep fusion merges feature hierarchies in an iterative and hierarchical manner, giving the network higher accuracy and fewer parameters.
  • the tree structure is used to replace the previous linear structure, which realizes the logarithmic level compression of the gradient return length of the network, instead of linear compression. In this way, the learned features are more descriptive and can effectively improve the prediction accuracy of the above numerical indicators.
  • the first image may be subjected to semantic segmentation processing to obtain semantic prediction results of multiple pixels in the first image, and an instance segmentation result of the first image may be determined based on the semantic prediction results of multiple pixels.
  • the semantic segmentation process is used to group (segment) pixels in the first image according to different semantic meanings. For example, it can be determined whether each of the multiple pixels included in the first image is an instance or a background, that is, is located in the instance area or the background area.
  • Pixel-level semantic segmentation can classify each pixel in the image into a corresponding category, that is, to achieve pixel-level classification; and the specific object of the class is an example. Instance segmentation not only needs to be classified at the pixel level, but also needs to distinguish different instances based on specific categories. For example, there are three nuclei 1, 2, and 3 in the first image. The semantic segmentation results are all nuclei, but the instance segmentation results are different objects.
  • an independent instance judgment may be performed for each pixel point in the first image, and a semantic segmentation category and an instance ID to which it belongs may be determined. For example, if there are three nuclei in an image, the semantic segmentation category of each nuclei is 1, but the IDs of different nuclei are 1, 2, and 3 respectively. Different nuclei can be distinguished by the aforementioned nuclei ID.
  • the semantic prediction results of the pixels may indicate that the pixels are located in the instance area or the background area. That is, the semantic prediction result of a pixel point indicates that the pixel point is an instance or a background.
  • the semantic prediction result of the pixel may include indication information for indicating whether the pixel is a cell nuclear region or a background region in the cell image.
  • the semantic prediction result of the pixel may be one of two preset values, and the two preset values respectively correspond to the instance area and the background area.
  • the semantic prediction result of a pixel may be 0 or a positive integer (for example, 1). Wherein, 0 represents a background area, and a positive integer (for example, 1) represents an example area, but embodiments of the present disclosure are not limited thereto.
  • the above semantic prediction result may be a binary result.
  • the first image may be processed to obtain an instance region prediction probability of each pixel point in the multiple pixel points, where the instance region prediction probability indicates a probability that the pixel point is located in the instance region.
  • the binning process is performed on the prediction probability of the instance region of each of the plurality of pixels to obtain a semantic prediction result of each of the plurality of pixels.
  • the second threshold value of the binarization process may be 0.5.
  • pixels with a prediction probability of the instance region greater than or equal to 0.5 are determined as pixels located in the instance region, and pixels with a prediction probability of the instance region less than 0.5 are determined as pixels located in the background region.
  • the semantic prediction result of pixels whose instance region prediction probability is greater than or equal to 0.5 may be determined as 1, and the semantic prediction result of pixels whose instance region prediction probability is less than 0.5 may be determined as 0, but embodiments of the present disclosure are not limited to this.
  • the prediction result of the pixel point may include the prediction result of the center relative position of the pixel point, which is used to indicate the relative position between the pixel point and the center of the instance to which the pixel point belongs.
  • the prediction result of the center relative position of the pixel point may include a prediction result of the center vector of the pixel point.
  • the prediction result of the relative position of the center of the pixel point can be expressed as a vector (x, y), which represents the difference between the coordinates of the pixel point and the coordinates of the center of the instance on the horizontal and vertical axes.
  • the prediction result of the relative position of the center of the pixel point may also be implemented in other manners, which is not limited in the embodiment of the present disclosure.
  • the instance center predicted position of the pixel that is, the predicted position of the center of the instance to which the pixel belongs, and the pixel based on the predicted position of the instance center of the pixel, to determine the pixel Point belongs to the example, but the embodiment of the present disclosure does not limit this.
  • position information of at least one instance center in the first image may be determined, and based on the predicted position of the instance center of the pixel and the position information of the at least one instance center, the pixel belongs to Instance.
  • a small area to which the instance center belongs can be defined as the instance center area.
  • the instance center area is an area within the instance area and smaller than the instance area, and the geometric center of the instance center area overlaps or is adjacent to the geometric center of the instance area, for example, the center of the instance center area is the instance center.
  • the instance's central area can be circular, oval, or other shapes. The above-mentioned instance central area can be set as required, and the embodiment of the present disclosure does not limit the specific implementation of the instance central area.
  • At this time, at least one instance center area in the first image may be determined, and an instance to which the pixel belongs may be determined based on a position relationship between the predicted position of the instance center of the pixel point and the at least one instance center area.
  • the specific implementation is not limited.
  • the prediction result of the pixel point may further include a prediction result of the central area of the pixel point, indicating whether the pixel point is located in the central area of the instance. Accordingly, at least one instance central region of the first image may be determined based on a prediction result of a central region of each of the plurality of pixel points.
  • the first image may be processed by a neural network to obtain a prediction result of a central area of each pixel among a plurality of pixels included in the first image.
  • the aforementioned neural network may be obtained by training through a supervised training method.
  • the sample images used in the training process can be labeled with instance information, and the central area of the instance can be determined based on the instance information labeled with the sample image, and the determined central area of the instance is used as a supervision to train the neural network.
  • the instance center may be determined based on the instance information, and an area containing a preset size or area of the instance center may be determined as the center area of the instance.
  • the sample image can also be etched to obtain the etched sample image, and the central region of the instance can be determined based on the etched sample image.
  • the corrosion operation of the image means that the image is detected with a certain structural element in order to find out the area where the structural element can be dropped inside the image.
  • the image etching process mentioned in the embodiment of the present disclosure may include the above-mentioned etching operation.
  • the etching operation is a process in which a structural element is translated and filled in the corroded image. From the results of the erosion, the foreground area of the image is reduced, and the boundary of the area is blurred. At the same time, some smaller isolated foreground areas are completely eroded, and the filtering effect is achieved.
  • each instance mask For each instance mask, first use a 5 ⁇ 5 convolution kernel to perform image erosion on the instance mask. Then, the coordinates of multiple pixel points included in the instance are averaged to obtain the center position of the instance, and the maximum distance from all the pixel points in the instance to the center position of the instance is determined, and the distance from the center position of the instance The pixels less than 30% of the maximum distance are determined as the pixels of the central area of the instance, that is, the central area of the instance is obtained. In this way, after the instance mask in the sample image is reduced by one circle, image binarization processing is performed to obtain a predicted binary image mask for the central region.
  • the center relative position information of the pixel point that is, the relative position information between the pixel point and the instance center, such as A vector to the center of the instance, and use this relative position information as a supervise to train the neural network, but embodiments of the present disclosure are not limited to this.
  • the first region image may be processed to obtain a prediction result of a central region of each of a plurality of pixel points included in the first image.
  • the first image may be processed to obtain a prediction probability of a central area of each pixel among the multiple pixels included in the first image, and the multiple pixels are based on a first threshold.
  • a binarization process is performed on the prediction probability of the central region of, to obtain the prediction result of the central region of each of the plurality of pixel points.
  • the predicted probability of the central region of the pixel point may refer to the probability that the pixel point is located in the central region of the instance. Pixels that are not located in the central area of the instance can be pixels in the background area or pixels in the instance area.
  • the binarization process may be a binarization process with a fixed threshold or a binarization process with an adaptive threshold.
  • a binarization process with an adaptive threshold For example, bimodal method, P-parameter method, iterative method and OTSU method.
  • the embodiment of the present disclosure does not limit the specific implementation of the binarization process.
  • the first threshold value or the second threshold value of the above binarization process may be preset or determined according to an actual situation, which is not limited in the embodiment of the present disclosure.
  • the prediction result of the central region of the pixel point can be obtained by judging the magnitude relationship between the prediction probability of the central region of the pixel point and the first threshold.
  • the first threshold may be 0.5.
  • the pixels with the predicted probability of the central region greater than or equal to 0.5 can be determined as the pixels located in the central region of the instance, and the pixels with the predicted probability of the central region less than 0.5 are determined as the pixels not located in the central region of the instance, thereby obtaining The prediction result of the central area of each pixel.
  • the central region prediction result of a pixel with a central region prediction probability of 0.5 or more is determined as 1, and the central region prediction result of a pixel with a central region prediction probability of less than 0.5 is determined as 0, but the embodiment of the present disclosure is not limited to this.
  • step 102 may be performed.
  • an instance segmentation result of the first image is determined based on a semantic prediction result and a center relative position prediction result of each pixel in the multiple pixels.
  • step 101 after obtaining the above-mentioned semantic prediction result and the above-mentioned center relative position prediction result, at least one pixel point located in the instance area and relative position information between the at least one pixel point and the instance center to which it belongs may be determined.
  • at least one first pixel located in the instance area may be determined from the multiple pixels; based on the first pixel, The center relative position prediction result determines the instance to which the first pixel belongs.
  • At least one first pixel point located in the instance area may be determined according to a semantic prediction result of each pixel point in the multiple pixel points. Specifically, a pixel point indicating that a semantic prediction result among a plurality of pixel points is located in the instance area is determined as the first pixel point.
  • the instance to which the pixel belongs can be determined according to the prediction result of the relative position of the center of the pixel.
  • the instance segmentation result of the first image includes the pixels included in each instance of at least one instance, in other words, the instance to which each pixel located in the instance region belongs.
  • Different instances can be distinguished by different instance identifications or labels (such as instance IDs).
  • the instance ID may be an integer greater than 0. For example, the instance ID of instance a is 1, the instance ID of instance b is 2, and the instance ID corresponding to the background is 0.
  • the instance identifier corresponding to each pixel in the multiple pixels included in the first image can be obtained, or the instance identifier of each first pixel in the first image can be obtained, that is, the pixel located in the background region does not have a corresponding instance identifier. This embodiment of the present disclosure does not limit this.
  • the semantic prediction result is a cell and the center vector representing the prediction result of the center relative position points to a center region, then this pixel point is assigned to the nucleus region (nuclear semantic region) of the cell . All the pixels are allocated according to the above steps, and the cell segmentation result can be obtained.
  • Nuclei segmentation in a digital microscope can extract high-quality morphological features of the nucleus, as well as computational pathological analysis of the nucleus. This information is an important basis for judging, for example, the grade of cancer, and the effectiveness of medications.
  • the Otsu algorithm and the waterline (also called watershed or watershed) threshold algorithm were commonly used to solve the problem of cell instance segmentation.
  • the above method is not effective.
  • Instance segmentation can rely on Convolutional Neural Network (CNN).
  • CNN Convolutional Neural Network
  • MaskRCNN MaskRCNN
  • FCN simple combed full convolutional network
  • a center vector representing a positional relationship of a pixel with respect to the center of an instance is used for modeling, so that instance segmentation in image processing has the advantages of high speed and high accuracy.
  • the above FCN shrinks some instances into boundary classes, and then uses a targeted post-processing algorithm to trim the prediction of the instance to which the boundary belongs.
  • center vector modeling can more accurately predict the boundary state of the nucleus based on the data, without the need for complicated professional post-processing algorithms.
  • the aforementioned MaskRCNN first extracts the image of each independent instance through a rectangle, and then performs the two-type prediction of the cell and the background.
  • center vector modeling does not have this kind of problem, but can obtain relatively accurate predictions for the nucleus boundary, thereby improving the overall prediction accuracy.
  • the embodiments of the present disclosure can be applied to clinical auxiliary diagnosis. After the doctor obtains a digitally scanned image of a patient's organ and tissue section, the doctor can input the image into the process in the embodiment of the present disclosure to obtain a pixel mask of each independent cell nucleus. Then, the doctor can calculate the cell density and cell morphology of the organ based on the pixel mask of each independent nucleus of the organ, and then draw a more accurate medical judgment.
  • the embodiment of the present disclosure determines an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each pixel in a plurality of pixel points included in the first image, so that instance segmentation in image processing can be provided with High speed and high precision.
  • FIG. 2 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure.
  • FIG. 2 is further optimized based on FIG. 1.
  • the main body performing the steps of the embodiments of the present disclosure may be the aforementioned electronic device.
  • the image processing method includes the following steps:
  • the second image is pre-processed to obtain a first image, so that the first image meets a preset contrast and / or a preset grayscale value.
  • the second image mentioned in the embodiment of the present disclosure may be a multi-modal pathological image obtained through various image acquisition devices (such as a microscope).
  • image acquisition devices such as a microscope.
  • the above multi-modality can be understood as that the image types can be diversified, and the characteristics such as image size, color, and resolution may be different, and the displayed image style is different, that is, the second image may be one or more.
  • the pathological image data obtained usually varies greatly.
  • the resolution of pathological images acquired under different microscopes can vary greatly. Light microscopy can obtain color images of pathological tissue (lower resolution), while electron microscopes can usually only capture grayscale images (but higher resolution).
  • the main body performing the steps of the embodiments of the present disclosure may be the aforementioned electronic device.
  • the electronic device may store the preset contrast and / or the preset gray value, and may convert the second image into a first image that meets the preset contrast and / or the preset gray value, and then execute Step 202.
  • the contrast ratio mentioned in the embodiment of the present disclosure refers to the measurement of different brightness levels between the brightest white and the darkest black in the light and dark areas in an image.
  • a larger difference range means a larger contrast
  • a smaller difference range means a contrast. The smaller.
  • each point on the black and white photo taken or the black and white image reproduced by the television receiver shows different degrees of gray.
  • the logarithmic relationship between white and black is divided into several levels, called “gray levels.”
  • the range of gray levels is generally from 0 to 255, white is 255, and black is 0. Therefore, black-and-white pictures are also called gray-scale images, which have a wide range of uses in the fields of medicine and image recognition.
  • the above preprocessing may also unify parameters such as the size, resolution, and format of the second image.
  • the second image may be cropped to obtain a first image of a preset image size, such as a first image of a uniform size of 256 * 256.
  • the electronic device may further store a preset image size and / or a preset image format, and may convert and obtain a first image that satisfies the preset image size and / or the preset image format during preprocessing.
  • Electronic devices can use technologies such as Image Super Resolution and image conversion to unify the multi-modality pathological images acquired by different pathological tissues and different imaging devices, so that they can be used as the image processing flow in the embodiments of the present disclosure.
  • input of. This step can also be called the image normalization process. Converting to a unified style image is more convenient for subsequent unified processing of the image.
  • Image super-resolution technology refers to a technology that uses image processing methods to convert existing low-resolution (LR) images into high-resolution (HR) images through software algorithms (emphasis is placed on unchanged imaging hardware equipment). Divided into super-resolution restoration and super-resolution image reconstruction (SRIR). At present, image super-resolution research can be divided into three main categories: interpolation-based, reconstruction-based, and learning-based methods. The core idea of super-resolution reconstruction is to exchange the temporal bandwidth (acquisition a sequence of multiple frames of the same scene) for the spatial resolution, and realize the conversion from temporal resolution to spatial resolution. Through the above preprocessing, a high-resolution first image can be obtained, which is very helpful for the doctor to make a correct diagnosis. If high-resolution images can be provided, the performance of pattern recognition in computer vision will also be greatly improved.
  • the first image is processed to obtain prediction results of multiple pixels in the first image.
  • the above prediction results include a semantic prediction result, a center relative position prediction result, and a center area prediction result.
  • the semantic prediction result indicates that the pixel point is located in the instance area or the background area
  • the center relative position prediction result indicates the relative position between the pixel point and the instance center
  • the central area prediction result indicates whether the pixel point is located in the instance center area.
  • step 202 reference may be made to the detailed description in step 101 of the embodiment shown in FIG. 1, and details are not described herein again.
  • At least one first pixel in the instance area is determined from the plurality of pixels.
  • each pixel Based on the semantic prediction results of each of the multiple pixels, it can be determined whether each pixel is located in the instance area or the background area, so that at least one first pixel located in the instance area can be determined from the multiple pixels .
  • At least one instance central area of the first image is determined based on a prediction result of a central area of each of the plurality of pixel points.
  • the central area of the example may refer to the specific description in the embodiment shown in FIG. 1, which is not repeated here.
  • the prediction result of the central region may indicate whether the pixel point is located in the central region of the instance, and thus the pixel point located in the central region of the instance may be determined by referring to the prediction result of the central region.
  • These pixels located in the center area of the instance can constitute the center area of the instance, and at least one instance center area can be determined.
  • a connected domain search process may be performed on the first image to obtain at least one instance central area.
  • the connected region generally refers to an image region (Region, Blob) composed of adjacent foreground pixels having the same pixel value in the image.
  • the above-mentioned connected domain search can be understood as connected area analysis (Connected Component Analysis), which is used to find and label each connected area in the image.
  • Connected area analysis is a more common and basic method in many application fields of the International Conference on Computer Vision and Pattern Recognition (CVPR) and image analysis and processing.
  • Optical character recognition Optical Character Recognition, OCR
  • character segmentation extraction (license plate recognition, text recognition, subtitle recognition, etc.), moving foreground target segmentation and extraction in visual tracking (pedestrian intrusion detection, residual object detection, vision-based Vehicle detection and tracking, etc.), medical image processing (target area of interest extraction), and so on.
  • the connected area analysis method can be used in any application scenario where the foreground target needs to be extracted for subsequent processing.
  • the object of the connected area analysis processing is a binary image (binary image). .
  • the condition that there is a path for the set S is that a certain arrangement of the pixels of the path makes the adjacent pixels meet a certain adjacency relationship. For example, suppose that there are A1, A2, A3,... An pixels between point p and point q, and that adjacent pixel points satisfy some kind of adjacency, then there is a path between p and q. If the pathway is connected end to end, it is called a closed pathway. There is only one path at a point p in the S set, which is called a connected component. If S has only one connected component, it is called a connected set.
  • R As a subset of images, if R is connected, then R is called a region. For all K areas that are not connected, the union Rk constitutes the foreground of the image, and the complement of Rk is called the background.
  • a connected domain search process is performed on the first image to obtain at least one instance central area, and then step 205 is performed.
  • a connected domain with a central area of 1 can be found to determine the instance central area, and an independent ID is assigned to each connected domain.
  • the pointing position of the center vector is in the center region based on the coordinates of a pixel in the cell nucleus and a center vector representing a position relationship of the pixel with respect to the center of the instance to which it belongs. If the center point of the pixel vector is in the center area, the nucleus ID is assigned to the pixel; otherwise, it indicates that the pixel does not belong to any nucleus and can be assigned nearby.
  • a random walk algorithm may be used to perform a connected domain search process on the first image to obtain at least one instance central area.
  • Random walk (also known as random walk, random walk, etc.) is based on past performance and cannot predict future development steps and directions.
  • the core concept of random walk is that the conserved quantity carried by any irregular walker corresponds to a diffusion transport law, which is close to Brownian motion, and is an ideal mathematical state of Brownian motion.
  • the basic idea of random walk for image processing in the embodiments of the present disclosure is to treat the image as a connected weighted undirected graph composed of fixed vertices and edges, start random walks from unlabeled vertices, and arrive for the first time
  • the probabilities of various types of labeled vertices represent the possibility that the unlabeled points belong to the labeled class.
  • the labels with the greatest probability are assigned to the unlabeled vertices to complete the segmentation.
  • the random walk algorithm described above can be used to allocate pixels that do not belong to any central area to obtain the at least one instance central area.
  • the pixel connection map can be output through the deep-level fusion network model, and the instance segmentation result can be obtained after the connected domain search processing. Random color can be given to each instance area in the above-mentioned instance segmentation results to facilitate visualization.
  • steps 203 and 204 may also be performed in no particular order; after determining the central area of the at least one instance, step 205 may be performed.
  • an instance center area corresponding to each of the first pixel points is determined from the at least one instance center area.
  • the center predicted position of the first pixel point may be determined based on the position information of the first pixel point and a center relative position prediction result of the first pixel point.
  • the position information of the pixels can be obtained, which can be specifically the coordinates of the pixels.
  • the center predicted position of the first pixel point may be determined.
  • the center prediction position may indicate a center position of an instance center area to which the predicted first pixel point belongs.
  • the instance center area corresponding to the first pixel point may be determined from the at least one instance center area.
  • the position information of the central area of the instance can be obtained, and it can also be represented by coordinates. Furthermore, based on the center prediction position of the first pixel point and the position information of at least one instance center area, it can be determined whether the center prediction position of the first pixel point belongs to the at least one instance center area, and thus from the at least one instance center area, Determines the instance central area corresponding to the first pixel.
  • the first instance center region is determined as the instance center region corresponding to the first pixel point, and Assign the pixel to the instance's center area.
  • the nearest allocation is performed, that is, the instance center area that is closest to the center prediction position of the first pixel point in the at least one instance center area It is determined as the instance central area corresponding to the first pixel point.
  • the output of the embodiment of the present disclosure in the above step 202 may have three branches: the first is a semantic judgment branch including 2 channels to output each pixel located in the instance area or the background area; the second is a central area branch containing 2 channels to output each pixel in the central or non-central area; the third is the center vector branch, including 2 channels, to output the relative position between each pixel and the center of the instance, which can include pixels The horizontal and vertical components of a vector whose points point to the geometric center of the instance to which they belong.
  • the example is a segmentation object in the first image, and may specifically be a closed structure in the first image.
  • the segmentation object may be a nucleus.
  • the above-mentioned central region is a central region of a cell nucleus, after the above-mentioned central region is determined, the position of the nucleus is actually initially determined, and each cell nucleus may be assigned a numerical number, that is, the above-mentioned instance ID.
  • the input second picture is a 3-channel picture of [height, width, 3].
  • three arrays of [height, width, 2] can be obtained in step 202, which are in turn each pixel's Semantic prediction probability, center region prediction probability and center relative position prediction result.
  • the threshold probability of the above-mentioned central region can be binarized with a threshold value of 0.5, and then the central region of each cell nucleus can be obtained through a connected domain search process, and an independent numerical number is assigned.
  • the numerical number assigned by each of the cells is The aforementioned example IDs are used to facilitate the differentiation of different nuclei.
  • the semantic prediction result of a pixel a has been determined to be the nucleus instead of the background (it is determined to belong to the semantic area of the nucleus), and the center vector of the pixel a has been obtained in step 202.
  • the center vector of the point a points to the first center area of the at least one instance center area obtained in step 204, which indicates that the pixel point a has a corresponding relationship with the first center area.
  • the pixel point a belongs to the nucleus A where the first central region is located, and the first central region is the central region of the nucleus A.
  • the nucleus and the image background can be segmented. All pixels that belong to the nucleus can be assigned, and the nucleus to which each pixel belongs, the center region of the nucleus, or the center of the nucleus to which it belongs Achieve more accurate segmentation of cells and obtain accurate instance segmentation results.
  • the center vector is used for modeling, so that accurate prediction can be obtained for the nucleus boundary, thereby improving the overall prediction accuracy.
  • the embodiment of the present disclosure can be applied to clinical auxiliary diagnosis.
  • FIG. 1 For a detailed description, refer to the embodiment shown in FIG. 1, and details are not described herein again.
  • the embodiment of the present disclosure obtains a first image by preprocessing the second image, and determines based on a semantic prediction result, a center area prediction result, and a center relative position prediction result of each pixel among a plurality of pixels included in the first image.
  • the instance central area corresponding to each first pixel point of the instance area can effectively achieve accurate segmentation of the instance, and can make the instance segmentation in image processing have the advantages of high speed and high accuracy.
  • FIG. 3 is a schematic diagram of a segmentation result of a cell instance according to an embodiment of the present disclosure.
  • the method in the embodiment of the present disclosure is used for processing, and has the characteristics of high speed and high accuracy.
  • FIG. 3 can facilitate a clearer understanding of the method in the embodiment shown in FIG. 1 and FIG. 2.
  • the existing indicators can be used to label the prediction indicators.
  • the semantic prediction result, the center area prediction result, and the center relative position prediction result in the foregoing embodiment are embodied in FIG.
  • a nucleus may include a nucleus semantic region and a nucleus central region.
  • the semantic label of the pixel is 1, it means that the pixel belongs to the nucleus, and 0 is the background of the image; if the center of the pixel is marked as 1, it means that the pixel is the center of the cell area.
  • the center vector of this pixel is labeled (0,0) and can be used as a reference for other pixels (such as pixel A and pixel D in the figure.
  • the determination of pixel A can also represent the determination of a cell nucleus).
  • Each pixel corresponds to a coordinate
  • the center vector label is the coordinate of the pixel relative to the pixel center of the nucleus.
  • the center vector of pixel B relative to pixel A is labeled (-5, -5), and
  • the center vector label of the pixel that belongs to the center is (0,0), such as pixel A and pixel D.
  • it can be determined that the pixel point B belongs to the nuclear region to which the pixel point A belongs that is, the pixel point B is allocated to the nuclear region to which the pixel point A belongs, but is not in the nuclear core region but in the nuclear semantics. within the area.
  • the electronic device includes a hardware structure and / or a software module corresponding to each function.
  • the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application of the technical solution and design constraints. Skilled artisans may use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the present disclosure.
  • the embodiments of the present disclosure may divide the functional units of the electronic device according to the foregoing method examples.
  • each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional unit. It should be noted that the division of the units in the embodiments of the present disclosure is schematic, and is only a logical function division. There may be another division manner in actual implementation.
  • the electronic device 400 includes a prediction module 410 and a segmentation module 420.
  • the prediction module 410 is configured to process a first image to obtain a prediction result of multiple pixels in the first image.
  • the prediction result includes a semantic prediction result and a center relative position prediction result, wherein the semantic prediction result indicates that the pixel point is located in an instance area or a background area, and the center relative position prediction result indicates that the pixel point and the instance center A relative position between the two;
  • the segmentation module 420 is configured to determine an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each of the plurality of pixel points.
  • the electronic device 400 may further include a pre-processing module 430 for pre-processing the second image to obtain the first image, so that the first image satisfies a preset contrast and / or a preset gray value.
  • the segmentation module 420 may include a first unit 421 and a second unit 422.
  • the first unit 421 is configured to, based on a semantic prediction result of each pixel in the plurality of pixels, from the plurality of pixels. Determine at least one first pixel point located in the instance area among the pixel points; the second unit 422 is configured to determine an instance to which each first pixel point belongs based on a center relative position prediction result of each first pixel point .
  • the prediction result may further include a center area prediction result, and the center area prediction result indicates whether the pixel point is located in an instance center area.
  • the segmentation module 420 further includes a third unit 423, configured to determine at least one instance center area of the first image based on a prediction result of a center area of each of the plurality of pixel points;
  • the second unit 422 is specifically configured to determine an instance center area corresponding to each first pixel point based on a center relative position prediction result of each first pixel point.
  • the third unit 423 may be specifically configured to perform a connected domain search process on the first image to obtain at least one instance central region based on a prediction result of a central region of each of the plurality of pixel points.
  • the second unit 422 may be specifically configured to: determine a center predicted position of the first pixel point based on the position information of the first pixel point and a center relative position prediction result of the first pixel point; based on the first pixel point A center predicted position of a pixel point and position information of the at least one instance center area are used to determine an instance center area corresponding to the first pixel point from the at least one instance center area.
  • the second unit 422 may be specifically configured to determine that the first instance center region is the first instance center region in response to the center predicted position of the first pixel point belonging to the first instance center region of the at least one instance center region.
  • the second unit 422 may be specifically configured to: in response to that the center predicted position of the first pixel point does not belong to any instance center area in the at least one instance center area, and associate the at least one instance center area with the The instance center area closest to the center prediction position of the first pixel point is determined as the instance center area corresponding to the first pixel point.
  • the prediction module 410 may include a probability prediction unit 411 and a judgment unit 412.
  • the probability prediction unit 411 is configured to process the first image to obtain respective centers of multiple pixels in the first image.
  • the judging unit 412 is configured to perform a binarization process on the respective center region prediction probabilities of the plurality of pixels based on a first threshold to obtain a center region of each of the plurality of pixels forecast result.
  • the prediction module 410 may be specifically configured to input a first image to a neural network for processing, and output prediction results of multiple pixels in the first image.
  • the center vector is used for modeling, so that accurate prediction can be obtained for the nucleus boundary, thereby improving the overall prediction accuracy.
  • the image processing method in the embodiments of FIG. 1 and FIG. 2 described above can be implemented, and the instance segmentation is performed by the center vector method. Moreover, it is not necessary for practitioners to have higher domain knowledge, and it is possible to obtain certain labeled data in any instance segmentation problem and then process it to obtain better results.
  • the electronic device 400 shown in FIG. 4 is implemented.
  • the electronic device 400 can determine an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each pixel among a plurality of pixels included in the first image. , Can make instance segmentation in image processing has the advantages of fast speed and high accuracy.
  • FIG. 5 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure. This method can be executed by any electronic device, such as a terminal device, a server, or a processing platform, which is not limited in the embodiments of the present disclosure. As shown in FIG. 5, the image processing includes the following steps.
  • the N sets of instance segmentation output data are the instance segmentation output results obtained by processing the images by N instance segmentation models, and the N sets of instance segmentation output data have different data structures, and the N is an integer greater than 1.
  • each pixel must be independently judged to determine its semantic category and instance ID. For example, there are three nuclei 1, 2, and 3 in the image. The semantic categories are all nuclei, but the result of instance segmentation is different.
  • Instance segmentation can also be implemented by instance segmentation algorithms, such as machine learning models such as instance segmentation algorithms based on support vector machines.
  • instance segmentation algorithms such as machine learning models such as instance segmentation algorithms based on support vector machines.
  • the embodiments of the present disclosure do not limit the specific implementation of the instance segmentation model.
  • instance segmentation output data the instance segmentation results (hereinafter referred to as instance segmentation output data) of each instance segmentation model in the N instance segmentation models can be obtained, that is, N sets of instance segmentation output data are obtained.
  • the N group instance split output data may be obtained from other devices, and the embodiment of the present disclosure does not limit the manner of obtaining the N group instance split output data.
  • the instance segmentation model Before using the instance segmentation model to process the image, you can also preprocess the image, such as contrast and / or grayscale adjustment, or one, or any number of operations such as cropping, horizontal and vertical flipping, rotation, scaling, noise removal, etc.
  • preprocess the image such as contrast and / or grayscale adjustment, or one, or any number of operations such as cropping, horizontal and vertical flipping, rotation, scaling, noise removal, etc.
  • the instance segmentation output data output by the N instance segmentation models may have different data structures or meanings.
  • the instance segmentation output data includes [height, width] data.
  • the instance ID is 0 to indicate the background, and different numbers greater than 0 indicate different instances.
  • the instance segmentation output data of the first instance segmentation model is a three-class probability map of [boundary, target, background].
  • the instance segmentation output data of the 2 instance segmentation models are the binary classification probability map of [boundary, background] and the binary classification map with the dimension [target, background]; the instance segmentation output data of the third instance segmentation model is [center area, Target class, background] three-class probability map, and so on.
  • Different instance segmentation models have different meanings of data output.
  • the method in the embodiment of the present disclosure can perform cross-instance segmentation model integration on the basis of this N group of instance segmentation output data with different data structures.
  • step 502 may be performed.
  • the output data is segmented based on the N sets of examples to obtain the integrated semantic data and integrated central area data of the image.
  • the integrated semantic data indicates the pixels located in the instance area in the image
  • the integrated central area data indicates the pixels located in the instance area in the image.
  • the electronic device may divide the output data of the N groups of instances and perform conversion processing to obtain integrated semantic data and integrated central area data of the image.
  • the above instance area can be understood as the area where the instance is in the image, that is, the area other than the background area, and the integrated semantic data can indicate the pixels in the image that are located in the instance area.
  • the above integrated semantic data may include a judgment result of pixels located in a cell nuclear region.
  • the above-mentioned integrated central area data may indicate pixels in the above-mentioned image that are located in the central area of the instance.
  • the semantic data and central area data of each instance segmentation model can be obtained, that is, a total of N sets of semantic data and N sets of central area data. Then, based on the semantic data and central area data of each instance segmentation model in the above N instance segmentation models, integration processing is performed to obtain the integrated semantic data and integrated central area data of the image.
  • the instance identification information (instance ID) corresponding to each pixel in the instance segmentation model can be determined, and then based on the corresponding values of each pixel in the instance segmentation model.
  • the instance identification information is used to obtain the semantic prediction value of each pixel in the instance segmentation model.
  • the semantic data of the example segmentation model includes a semantic prediction value of each pixel among multiple pixels of the image.
  • Thresholding is a simple method for image segmentation.
  • Binarization can convert a grayscale image into a binary image. For example, the grayscale value of a pixel point greater than a certain threshold grayscale value can be set to a maximum grayscale value, and the grayscale value of a pixel point less than this value can be set to a minimum grayscale value, thereby achieving binarization.
  • a semantic prediction result of each pixel in a plurality of pixels included in the first image may be obtained by processing the first image.
  • the semantic prediction result of the pixel point can be obtained by judging the magnitude relationship between the semantic prediction value of the pixel point and the first threshold.
  • the foregoing first threshold may be preset or determined according to an actual situation, which is not limited in the embodiment of the present disclosure.
  • step 503 may be performed.
  • the at least one instance central area of the image may be obtained based on the integrated central area data of the image. Then, based on the integrated semantic data of the central area of the at least one instance and the image, an instance to which each pixel of the multiple pixels of the image belongs may be determined.
  • the above-mentioned integrated semantic data indicates at least one pixel point located in the instance area in the image.
  • the integrated semantic data may include an integrated semantic value of each pixel in a plurality of pixels of the image, and the integrated semantic value is used to indicate whether the pixel is located in the instance area or used to indicate that the pixel is located in the instance area or the background area.
  • the above-mentioned integrated central area data indicates at least one pixel point in the above-mentioned image located in the central area of the instance.
  • the integrated center area data includes an integrated center area prediction value for each pixel in a plurality of pixel points of the image, and the integrated center area prediction value is used to indicate whether the pixel point is located in the instance center area.
  • At least one pixel point included in the instance area of the image may be determined through the above integrated semantic data, and at least one pixel point included in the instance center area of the image may be determined through the above-mentioned integrated central area data. Based on the integrated central area data and integrated semantic data of the image, the instance to which each pixel of the multiple pixels of the image belongs can be determined, and the instance segmentation result of the image can be obtained.
  • the instance segmentation results obtained by the above method integrate the instance segmentation output results of N instance segmentation models, integrate the advantages of different instance segmentation models, no longer require different instance segmentation models to have the same meaning of data output, and improve the accuracy of instance segmentation .
  • the embodiment of the present disclosure obtains the integrated semantic data and integrated central area data of the above image based on N sets of instance segmented output data obtained by processing the image through the N instance segmentation models, and further based on the integrated semantic data and integrated central area of the above image.
  • the advantages of each instance segmentation model can be complemented without requiring each model to have data output with the same structure or meaning, and higher accuracy can be achieved in the instance segmentation problem.
  • FIG. 6 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure.
  • FIG. 6 is further optimized based on FIG. 5.
  • This method can be executed by any electronic device, such as a terminal device, a server, or a processing platform, which is not limited in the embodiments of the present disclosure.
  • the image processing method includes the following steps:
  • N sets of instance segmentation output data are obtained.
  • the N sets of instance segmentation output data are the instance segmentation output results obtained by processing the images by N instance segmentation models, and the N sets of instance segmentation output data have different data structures, and the N is an integer greater than 1.
  • step 601 reference may be made to the detailed description in step 501 of the embodiment shown in FIG. 5, and details are not described herein again.
  • the instance segmentation output data may include instance identification information corresponding to each pixel in at least two pixels in the instance area in the image, for example, the instance ID is an integer greater than 0, such as 1, 2, or 3, or it may be another value .
  • the instance identification information corresponding to the pixels located in the background area may be a preset value, or the pixels located in the background area may not correspond to any instance identification information. In this way, based on the instance identification information corresponding to each pixel point in the multiple pixel points in the output data of the instance segmentation, at least two pixel points located in the instance area in the image can be determined.
  • the instance segmentation output data may not include instance identification information corresponding to each pixel.
  • the instance segmentation output data can be processed to obtain at least two pixels in the image in the instance area, which is not limited in the embodiment of the present disclosure.
  • step 603 may be performed.
  • position information of the above at least two pixels can be obtained.
  • the position information may include coordinates of pixels in the image, but the embodiment of the present disclosure is not limited thereto.
  • the instance center position of the instance segmentation model may be determined according to the position information of the at least two pixels.
  • the above-mentioned instance center position is not limited to the geometric center position of the instance, but may be the predicted center position of the instance area, which can be understood as any position in the above-mentioned instance center area.
  • the average value of the positions of at least two pixels located in the instance area may be used as the instance center position of the instance segmentation model.
  • the coordinates of the at least two pixel points located in the instance area may be averaged and used as the coordinates of the instance center position of the instance segmentation model to determine the instance center position.
  • a maximum distance between the at least two pixel points and the instance center position may be determined, and then a first threshold value is determined based on the maximum distance. Then, a pixel point whose distance between the at least two pixel points and the center position of the instance is less than or equal to the first threshold may be determined as a pixel point in the center region of the instance.
  • the distance (pixel distance) from each pixel to the instance center position can be calculated.
  • the electronic device may set the algorithm of the first threshold in advance. For example, the first threshold may be set to 30% of the maximum distance among the pixel distances. After determining the maximum distance among the pixel point distances, the above-mentioned first threshold value may be calculated and obtained. Based on this, the pixel points whose pixel distance is less than the first threshold are determined, and these pixel points are determined as the pixel points of the central area of the instance, that is, the central area of the instance is determined.
  • the sample image can also be etched.
  • For the corrosion treatment reference may be made to the detailed description in the embodiment shown in FIG. 1, and details are not described herein again.
  • the electronic device may perform a semantic vote on each pixel in a plurality of pixels based on the semantic data of each instance segmentation model in the above N instance segmentation models, and determine the semantic vote of each pixel in the multiple pixels of the image value. For example, a sliding window-based voting may be used to process the semantic data of the above-mentioned example segmentation model to determine the semantic voting value of each pixel, and then step 606 may be performed.
  • the integrated semantic data of the image includes an integrated semantic value of each pixel in the multiple pixels.
  • Binary processing can be performed on the semantic voting values of the above N instance segmentation models for each pixel to obtain the integrated semantic value of each pixel in the image. It can be understood that the semantic masks obtained by different instance segmentation models are added to obtain an integrated semantic mask.
  • a second threshold value may be determined based on the number N of the multiple instance segmentation models; based on the second threshold value, the semantic voting value of each pixel in the multiple pixel points is binarized to obtain the foregoing.
  • the integrated semantic value of each pixel in the image may be determined based on the number N of the multiple instance segmentation models; based on the second threshold value, the semantic voting value of each pixel in the multiple pixel points is binarized to obtain the foregoing.
  • the second threshold may be determined based on the number N of the multiple instance segmentation models.
  • the second threshold may be a round-up result of N / 2.
  • the integrated semantic value of each pixel in the image can be obtained by using the second threshold as the judgment basis of the binarization process in this step.
  • the semantic voting value and the second threshold are compared.
  • the truncation of the semantic voting value of 2 or more is 1 and the truncation of the semantic voting value is less than 2 to obtain the integrated semantic value of each pixel in the image.
  • the output is
  • the data can be an integrated semantic binary map.
  • the above integrated semantic value can be understood as the result of the semantic segmentation of each pixel, and the instance to which the pixel belongs can be determined on the basis of this to implement instance segmentation.
  • a random walk is used to determine the distribution of the pixels according to the integrated semantic value of the pixels, so as to obtain the above-mentioned each The instance to which each pixel belongs. For example, the instance corresponding to the central area of the instance closest to the pixel may be determined as the instance to which the pixel belongs.
  • the embodiment of the present disclosure can determine the pixel allocation of an instance by obtaining the final integrated semantic map and integrated central area map, combined with a specific implementation of the above-mentioned connected area search and random walk (closest allocation), to obtain the final instance segmentation result.
  • the instance segmentation results obtained by the above method integrate the instance segmentation output results of N instance segmentation models, integrate the advantages of these instance segmentation models, no longer require different instance segmentation models to have continuous probability map output with the same meaning, and improve the instance Segmentation accuracy.
  • the method in the embodiment of the present disclosure is applicable to the problem of arbitrary instance segmentation.
  • it may be applied to clinical auxiliary diagnosis, and reference may be made to the detailed description in the embodiment shown in FIG. 1, and details are not described herein again.
  • Another example is that around the hive, after the breeder has obtained dense bees flying around the hive, he can use this algorithm to obtain an instance pixel mask of each independent bee. It can perform macro bee counting and behavior pattern calculation. Great practical value.
  • a UNet model may be preferably applied.
  • UNet was first developed for semantic segmentation and effectively fuses information from multiple scales.
  • a MaskR-CNN model can be applied.
  • MaskR-CNN extends the faster R-CNN by adding a head to the segmentation task.
  • the proposed MaskR-CNN can align the tracking features with the input, avoiding any quantization of bilinear interpolation. Alignment is important for pixel-level tasks, such as instance segmentation tasks.
  • the network structure of the UNet model consists of a contracting path and an expanding path.
  • the contraction path is used to obtain context information
  • the expansion path is used for precise localization
  • the two paths are symmetrical to each other.
  • the network can be trained end-to-end from very few images, and it performs better than previous best methods (sliding window convolutional network) for segmenting cell structures such as neurons in the electron microscope. In addition, it runs very fast,
  • UNet and Mask R-CNN models can be used to perform segmentation prediction on instances, to obtain the semantic mask of each instance segmentation model, and to integrate by pixel voting (Vote). Then, the center mask of each instance segmentation model is calculated through the erosion process, and the center mask is integrated. Finally, the random walk algorithm is used to obtain the instance segmentation results from the integrated semantic mask and center mask.
  • Cross-validation can be used to evaluate the above results.
  • Cross validation is mainly used in modeling applications. In a given modeling sample, take out most of the samples to build a model, leave a small part of the sample to use the model just established to forecast, and find the forecast error of this small sample, and record their sum of squares.
  • the embodiment of the present disclosure can be evaluated by 3 times cross-validation, combining three UJI models with AJI (5) scores of 0.605, 0.599, and 0.589 and one MaskR-CNN model with AJI (5) score of 0.565.
  • the result obtained by the method has a final AJI (5) score of 0.616, which shows that the image processing method of the present disclosure has obvious advantages.
  • the embodiments of the present disclosure determine instance central regions of the instance segmentation model based on instance segmentation output data obtained by processing an image using N instance segmentation models, and based on the integrated semantics of each pixel in a plurality of pixels of the image Value and at least one instance of the central area of the random walk to obtain the instance to which each pixel belongs, can achieve the complementary advantages of each instance segmentation model, no longer require each model to have the same structure or meaning of data output, segmentation in the instance Achieve higher accuracy in the problem.
  • FIG. 7 is a schematic diagram of an image representation of a cell instance segmentation according to an embodiment of the present disclosure.
  • a more accurate instance segmentation result can be obtained.
  • Use N types of instance segmentation models (only four are shown in the figure) to give instance prediction masks for the input picture (different colors represent different cell instances in the picture), and convert the instance prediction masks into semantic masks using semantic prediction segmentation
  • the pixel voting is performed separately, and then integration is performed to finally obtain the instance segmentation result.
  • the electronic device includes a hardware structure and / or a software module corresponding to each function.
  • the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application of the technical solution and design constraints. Skilled artisans may use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the present disclosure.
  • the embodiments of the present disclosure may divide the functional units of the electronic device according to the foregoing method examples.
  • each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional unit. It should be noted that the division of the units in the embodiments of the present disclosure is schematic, and is only a logical function division. There may be another division manner in actual implementation.
  • the electronic device 800 includes: an acquisition module 810, a conversion module 820, and a segmentation module 830.
  • the acquisition module 810 is configured to acquire N sets of instance segmentation output data, where the N set of instance segments
  • the output data is an instance segmentation output result obtained by processing the image by N instance segmentation models, and the N sets of instance segmentation output data have different data structures, where N is an integer greater than 1;
  • the conversion module 820 Configured to segment output data based on the N sets of instances to obtain integrated semantic data and integrated central area data of the image, wherein the integrated semantic data indicates pixels in the image that are located in the instance area, and the integrated central area
  • the data indicates pixels in the image located in the central area of the instance;
  • the segmentation module 830 is configured to obtain an instance segmentation result of the image based on the integrated semantic data and the integrated central area data of the image.
  • the conversion module 820 may include a first conversion unit 821 and a second conversion unit 822, where the first conversion unit 821 is configured to segment output data based on an instance of each instance segmentation model among the N instance segmentation models.
  • the second conversion unit 822 is configured to use the semantic data and the central region data of each instance segmentation model based on the N instance segmentation models to obtain The image's integrated semantic data and integrated central area data.
  • the first conversion unit 821 may be specifically configured to determine instance identification information corresponding to each pixel in multiple pixels of the image in the instance segmentation model based on instance segmentation output data of the instance segmentation model; Obtaining the semantic prediction value of each pixel in the instance segmentation model based on instance identification information corresponding to each pixel in the plurality of pixels in the instance segmentation model, wherein the instance segmentation model
  • the semantic data of Ai includes semantic predictive values of each pixel in a plurality of pixels of the image.
  • the first conversion unit 821 may be further specifically configured to: based on the instance segmentation output data of the instance segmentation model, determine in the instance segmentation model that at least two pixels in the image are located in the instance area; based on the Determining the instance center position of the instance segmentation model by using position information of at least two pixels in the instance segmentation model; based on the instance center position of the instance segmentation model and the position information of the at least two pixels, An instance central area of the instance segmentation model is determined.
  • the conversion module 820 may further include an erosion processing unit 823, configured to perform an erosion process on the instance segmentation output data of the instance segmentation model to obtain the corrosion data of the instance segmentation model.
  • the first conversion unit 821 may be specifically configured to The corrosion data of the instance segmentation model determines that in the instance segmentation model, at least two pixels in the image are located in an instance area.
  • the first conversion unit 821 may be specifically configured to use an average value of the positions of at least two pixels located in the instance area as an instance center position of the instance segmentation model.
  • the first conversion unit 821 may be further specifically configured to determine a maximum of the at least two pixels and the instance center position based on the instance center position of the instance segmentation model and the position information of the at least two pixels. Distance; determining a first threshold value based on the maximum distance; determining a pixel distance between the at least two pixel points and the instance center position that is less than or equal to the first threshold value as a pixel of the instance center area point.
  • the conversion module 820 may be specifically configured to determine a semantic voting value of each pixel in a plurality of pixels of the image based on the semantic data of each instance segmentation model in the N instance segmentation models; The semantic voting value of each pixel in the two pixels is binarized to obtain the integrated semantic value of each pixel in the image, and the integrated semantic data of the image includes each of the multiple pixels. Pixel's integrated semantic value.
  • the conversion module 820 may be further configured to determine a second threshold based on the number N of the multiple instance segmentation models; and based on the second threshold, perform semantics on each pixel of the multiple pixels.
  • the voting value is binarized to obtain the integrated semantic value of each pixel in the image.
  • the second threshold may be a round-up result of N / 2.
  • the segmentation module 830 may include a central area unit 831 and a determination unit 832, wherein: the central area unit 831 is configured to obtain at least one instance central area of the image based on the integrated central area data of the image; The determining unit 832 is configured to determine, based on the integrated semantic data of the at least one instance central area and the image, an instance to which each pixel of the multiple pixels of the image belongs.
  • the determining unit 832 may be specifically configured to perform a random walk based on the integrated semantic value of each pixel in the multiple pixels of the image and the at least one instance center region to obtain the Instance.
  • the electronic device 800 shown in FIG. 8 is implemented.
  • the electronic device 800 can segment output data based on N sets of instances obtained by processing images through N instance segmentation models to obtain integrated semantic data and integrated central area data of the above images, and then based on the The integrated semantic data of the image and the central region data are used to obtain the instance segmentation results of the above image, which can achieve the complementary advantages of each instance segmentation model, instead of requiring that each model have the same structure or meaning data output. Higher accuracy.
  • FIG. 9 is a schematic structural diagram of another electronic device according to an embodiment of the present disclosure.
  • the electronic device 900 includes a processor 901 and a memory 902.
  • the electronic device 900 may further include a bus 903.
  • the processor 901 and the memory 902 may be connected to each other through the bus 903.
  • the bus 903 may be a Peripheral Component Interconnect (PCI) bus or an extended industry standard structure (Extended Industry). Standard Architecture (EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Standard Architecture
  • the bus 903 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only a thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.
  • the electronic device 900 may further include an input-output device 904, and the input-output device 904 may include a display screen, such as a liquid crystal display screen.
  • the memory 902 is configured to store a computer program; the processor 901 is configured to call the computer program stored in the memory 902 to execute some or all of the method steps mentioned in the embodiments of FIG. 1, FIG. 2, FIG. 5, and FIG. 6.
  • the electronic device 900 shown in FIG. 9 is implemented.
  • the electronic device 900 can determine an instance segmentation result of the first image based on a semantic prediction result and a center relative position prediction result of each pixel among a plurality of pixels included in the first image. , Can make instance segmentation in image processing has the advantages of fast speed and high accuracy.
  • the electronic device 900 shown in FIG. 9 is implemented.
  • the electronic device 900 can segment output data based on N sets of instances obtained by processing images through N instance segmentation models to obtain integrated semantic data and integrated central area data of the above images, and then based on the above.
  • the integrated semantic data of the image and the central area data are used to obtain the instance segmentation results of the above image.
  • the advantages of each instance segmentation model can be achieved. It is no longer required that each model has the same structure or meaning of data output. High accuracy.
  • An embodiment of the present disclosure also provides a computer storage medium, wherein the computer storage medium is used to store a computer program, and the computer program causes a computer to execute part or all of the steps of any one of the image processing methods described in the foregoing method embodiments.
  • the disclosed device may be implemented in other manners.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or may be combined. Integration into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical or other forms.
  • the units (modules) described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place, or may be distributed to multiple networks On the unit. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
  • the integrated unit When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable memory.
  • the technical solution of the present disclosure essentially or part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory.
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • the foregoing memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk, and other media that can store program codes.
  • the program may be stored in a computer-readable memory, and the memory may include a flash disk. , Read-only memory, random access device, disk or optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

一种图像处理方法、电子设备及存储介质。根据方法的一个示例，电子设备可对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实例中心之间的相对位置(101)；基于所述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定所述第一图像的实例分割结果(102)。

Description

图像处理方法、电子设备及存储介质
相关申请
本公开要求在2018年9月15日提交中国专利局、申请号为201811077349.X、申请名称为“一种图像处理方法、电子设备及存储介质”以及2018年9月15日提交中国专利局、申请号为201811077358.9、申请名称为“一种图像处理方法、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及计算机视觉技术领域，具体涉及图像处理方法、电子设备及存储介质。
背景技术
影像处理又称为图像处理，是用计算机对图像进行分析，以达到所需结果的技术。图像处理一般指数字图像处理。其中，数字图像是指用工业相机、摄像机、扫描仪等设备拍摄得到的一个二维数组，该数组的元素称为像素点，其值称为灰度值。图像处理在许多领域起着十分重要的作用，尤其是对医学影像的处理。
发明内容
本公开实施例提供了一种图像处理方法、电子设备及存储介质。
本公开实施例第一方面提供一种图像处理方法，包括：对第一图像进行处理，获得所述第一图像中多个像素点各自的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实例中心之间的相对位置；基于所述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定所述第一图像的实例分割结果。
可选的，所述对第一图像进行处理，获得所述第一图像中多个像素点的语义预测结果包括：对所述第一图像进行处理，得到所述第一图像中多个像素点的实例区域预测概率，所述实例区域预测概率指示该像素点位于实例区域的概率；基于第二阈值对上述多个像素点的实例区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的语义预测结果。
可选的，所述实例中心区域包括：在所述实例区域内并且小于所述实例区域的区域，并且所述实例中心区域的几何中心与所述实例区域的几何中心重叠。
在一种可选的实施方式中，在对第一图像进行处理之前，所述方法还包括：对第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设对比度和/或预设灰度值。
在一种可选的实施方式中，在对所述第一图像进行处理之前，所述方法还包括：对所述第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设图像大小。
在一种可选的实施方式中，所述基于所述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定所述第一图像的实例分割结果，包括：基于所述多个像素点中每个像素点的语义预测结果，从所述多个像素点中确定位于实例区域的至少一个 第一像素点；针对每个所述第一像素点，基于所述第一像素点的中心相对位置预测结果，确定所述第一像素点所属的实例。
所述实例为第一图像中的分割对象，具体可以为第一图像中的封闭性结构。
本公开实施例中的实例包括细胞核，即本公开实施例可以应用于细胞核分割。
在一种可选的实施方式中，所述预测结果还包括：中心区域预测结果，所述中心区域预测结果指示所述像素点是否位于实例中心区域。在此情况下，所述方法还包括：基于所述多个像素点中每个像素点的中心区域预测结果，确定所述第一图像的至少一个实例中心区域；所述基于所述第一像素点的中心相对位置预测结果，确定所述第一像素点所属的实例，包括：基于所述第一像素点的中心相对位置预测结果，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述基于所述多个像素点中每个像素点的中心区域预测结果，确定所述第一图像的至少一个实例中心区域，包括：基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
在一种可选的实施方式中，所述基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域包括：基于所述多个像素点中每个像素点的中心区域预测结果，使用随机游走算法对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
在一种可选的实施方式中，基于所述第一像素点的中心相对位置预测结果，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域，包括：基于所述第一像素点的位置信息和所述第一像素点的中心相对位置预测结果，确定所述第一像素点的中心预测位置；基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域，包括：响应于所述第一像素点的中心预测位置属于所述至少一个实例中心区域中的第一实例中心区域，将所述第一实例中心区域确定为所述第一像素点对应的实例中心区域；或者，响应于所述第一像素点的中心预测位置不属于所述至少一个实例中心区域中的任意实例中心区域，将所述至少一个实例中心区域中与所述第一像素点的中心预测位置距离最近的实例中心区域确定为所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，包括：对所述第一图像进行处理，得到所述第一图像中多个像素点的中心区域预测概率；基于第一阈值对所述多个像素点的中心区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的中心区域预测结果。
在一种可选的实施方式中，所述对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，包括：将第一图像输入到神经网络进行处理，输出所述第一图像中多个像素点的预测结果。
本公开实施例第二方面提供一种电子设备，包括预测模块和分割模块，其中：所述预测模块，用于对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实 例中心之间的相对位置；所述分割模块，用于基于所述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定所述第一图像的实例分割结果。
可选的，所述预测模块具体用于：对所述第一图像进行处理，得到所述第一图像中多个像素点的实例区域预测概率，所述实例区域预测概率指示该像素点位于实例区域的概率；基于第二阈值对上述多个像素点的实例区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的语义预测结果。
在一种可选的实施方式中，所述电子设备还包括预处理模块，用于对第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设对比度和/或预设灰度值。
在一种可选的实施方式中，所述预处理模块，还用于对所述第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设图像大小。
在一种可选的实施方式中，所述分割模块包括第一单元和第二单元，其中：所述第一单元用于基于所述多个像素点中每个像素点的语义预测结果，从所述多个像素点中确定位于实例区域的至少一个第一像素点；所述第二单元用于基于所述至少一个第一像素点中每个第一像素点的中心相对位置预测结果，确定所述每个第一像素点所属的实例。
在一种可选的实施方式中，所述预测结果还包括中心区域预测结果，所述中心区域预测结果指示所述像素点是否位于实例中心区域，所述分割模块还包括第三单元，用于基于所述多个像素点中每个像素点的中心区域预测结果，确定所述第一图像的至少一个实例中心区域；所述第二单元具体用于，基于所述至少一个第一像素点中每个第一像素点的中心相对位置预测结果，从所述至少一个实例中心区域中确定所述每个第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述第三单元具体用于，基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
在一种可选的实施方式中，所述第三单元具体用于，基于所述多个像素点中每个像素点的中心区域预测结果，使用随机游走算法对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
在一种可选的实施方式中，所述第二单元具体用于：基于所述第一像素点的位置信息和所述第一像素点的中心相对位置预测结果，确定所述第一像素点的中心预测位置；基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述第二单元具体用于：响应于所述第一像素点的中心预测位置属于所述至少一个实例中心区域中的第一实例中心区域，将所述第一实例中心区域确定为所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述第二单元具体用于：响应于所述第一像素点的中心预测位置不属于所述至少一个实例中心区域中的任意实例中心区域，将所述至少一个实例中心区域中与所述第一像素点的中心预测位置距离最近的实例中心区域确定为所述第一像素点对应的实例中心区域。
在一种可选的实施方式中，所述预测模块包括概率预测单元和判断单元，其中：所述概率预测单元，用于对所述第一图像进行处理，得到所述第一图像中多个像素点的中心区域预测概率；所述判断单元，用于基于第一阈值对所述多个像素点的中心区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的中心区域预测结果。
在一种可选的实施方式中，所述预测模块具体用于，将第一图像输入到神经网络进行处理，输出所述第一图像中多个像素点的预测结果。
本公开实施例中，通过基于第一图像包含的多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定上述第一图像的实例分割结果，可以使图像处理中的实例分割具备速度快、精度高的优点。
本公开实施例第三方面提供一种图像处理方法，包括：获取N组实例分割输出数据，其中，所述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且所述N组实例分割输出数据具有不同的数据结构，所述N为大于1的整数；基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，其中，所述集成语义数据指示所述图像中位于实例区域的像素点，所述集成中心区域数据指示所述图像中位于实例中心区域的像素点；基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果。
在一种可选的实施方式中，所述基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，包括：针对所述N个实例分割模型中每个实例分割模型，基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据；基于所述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据，得到所述图像的集成语义数据和集成中心区域数据。
在一种可选的实施方式中，所述基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据，包括：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中所述图像的多个像素点中每个像素点对应的实例标识信息；基于所述实例分割模型中所述多个像素点中每个像素点对应的实例标识信息，得到所述每个像素点在所述实例分割模型中的语义预测值，其中，所述实例分割模型的语义数据包括所述图像的多个像素点中每个像素点的语义预测值。
在一种可选的实施方式中，所述基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据，还包括：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点；基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定所述实例分割模型的实例中心位置；基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域。
在一种可选的实施方式中，在基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点之前，还包括：对所述实例分割模型的实例分割输出数据进行腐蚀处理，得到实例分割模型的腐蚀数据。在此情况下，所述基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点，包括：基于所述实例分割模型的腐蚀数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点。
在一种可选的实施方式中，所述基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定所述实例分割模型的实例中心位置，包括：将所述位于实例区域的至少两个像素点的位置的平均值，作为所述实例分割模型的实例中心位置。
在一种可选的实施方式中，所述基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域，包括：基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述至少两个像素点与所述实例中心位置的最大距离；基于所述最大距离，确定第一阈值；将所述至少两个 像素点中与所述实例中心位置之间的距离小于或等于所述第一阈值的像素点确定为实例中心区域的像素点。
在一种可选的实施方式中，所述基于所述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据，得到所述图像的集成语义数据和集成中心区域数据，包括：基于所述N个实例分割模型中每个实例分割模型的语义数据，确定所述图像的多个像素点中每个像素点的语义投票值；对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值，其中，所述图像的集成语义数据包括所述多个像素点中每个像素点的集成语义值。
在一种可选的实施方式中，所述对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值，包括：基于所述多个实例分割模型的个数N，确定第二阈值；基于所述第二阈值，对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值。
在一种可选的实施方式中，所述第二阈值为N/2的向上取整结果。
在一种可选的实施方式中，所述基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果，包括：基于所述图像的集成中心区域数据，得到所述图像的至少一个实例中心区域；基于所述至少一个实例中心区域和所述图像的集成语义数据，确定所述图像的多个像素点中每个像素点所属的实例。
在一种可选的实施方式中，所述基于所述至少一个实例中心区域和所述图像的集成语义数据，确定所述图像的多个像素点中每个像素点所属的实例，包括：基于所述图像的多个像素点中每个像素点的集成语义值和所述至少一个实例中心区域，进行随机游走，得到所述每个像素点所属的实例。
本公开实施例第四方面提供一种电子设备，包括：获取模块、转换模块和分割模块，其中：所述获取模块，用于获取N组实例分割输出数据，其中，所述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且所述N组实例分割输出数据具有不同的数据结构，所述N为大于1的整数；所述转换模块，用于基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，其中，所述集成语义数据指示所述图像中位于实例区域的像素点，所述集成中心区域数据指示所述图像中位于实例中心区域的像素点；所述分割模块，用于基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果。
在一种可选的实施方式中，所述转换模块包括第一转换单元和第二转换单元，其中：所述第一转换单元，用于针对所述N个实例分割模型中每个实例分割模型，基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据；所述第二转换单元，用于基于所述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据，得到所述图像的集成语义数据和集成中心区域数据。
在一种可选的实施方式中，所述第一转换单元具体用于：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中所述图像的多个像素点中每个像素点对应的实例标识信息；基于所述实例分割模型中所述多个像素点中每个像素点对应的实例标识信息，得到所述每个像素点在所述实例分割模型中的语义预测值，其中，所述实例分割模型的语义数据包括所述图像的多个像素点中每个像素点的语义预测值。
在一种可选的实施方式中，所述第一转换单元具体还用于：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点；基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定 所述实例分割模型的实例中心位置；基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域。
在一种可选的实施方式中，所述转换模块还包括腐蚀处理单元，用于对所述实例分割模型的实例分割输出数据进行腐蚀处理，得到实例分割模型的腐蚀数据；所述第一转换单元具体用于，基于所述实例分割模型的腐蚀数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点。
在一种可选的实施方式中，所述第一转换单元具体用于，将所述位于实例区域的至少两个像素点的位置的平均值，作为所述实例分割模型的实例中心位置。
在一种可选的实施方式中，所述第一转换单元具体还用于：基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述至少两个像素点与所述实例中心位置的最大距离；基于所述最大距离，确定第一阈值；将所述至少两个像素点中与所述实例中心位置之间的距离小于或等于所述第一阈值的像素点确定为实例中心区域的像素点。
在一种可选的实施方式中，所述转换模块，具体用于：基于所述实例分割模型的语义数据，确定所述图像的多个像素点中每个像素点的语义投票值；对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值，其中，所述图像的集成语义数据包括所述多个像素点中每个像素点的集成语义值。
在一种可选的实施方式中，所述转换模块，具体还用于：基于所述多个实例分割模型的个数N，确定第二阈值；基于所述第二阈值，对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值。
在一种可选的实施方式中，所述第二阈值为N/2的向上取整结果。
本公开实施例第五方面提供另一种电子设备，包括处理器以及存储器，所述存储器用于存储计算机程序，所述计算机程序被配置成由所述处理器执行，所述处理器用于执行如本公开实施例第一方面和第三方面任一方法中所描述的部分或全部步骤。
本公开实施例第六方面提供一种计算机可读存储介质，所述计算机可读存储介质用于存储计算机程序，其中，所述计算机程序使得计算机执行如本公开实施例第一方面和第三方面任一方法中所描述的部分或全部步骤。
本公开实施例基于通过N个实例分割模型对图像进行处理获得的N组实例分割输出数据，得到上述图像的集成语义数据和集成中心区域数据，进而基于上述图像的集成语义数据和集成中心区域数据，获得上述图像的实例分割结果，可以实现各个实例分割模型的优势互补，不再要求各个模型具有相同结构或含义的数据输出，在实例分割问题中取得更高的精度。
附图说明
图1是本公开实施例的一种图像处理方法的流程示意图；
图2是本公开实施例的另一种图像处理方法的流程示意图；
图3是本公开实施例的一种细胞实例分割结果示意图；
图4是本公开实施例的一种电子设备的结构示意图；
图5是本公开实施例的又一种图像处理方法的流程示意图。
图6是本公开实施例的再一种图像处理方法的流程示意图。
图7是本公开实施例的一种细胞实例分割的图像表现形式示意图。
图8是本公开实施例的另一种电子设备的结构示意图。
图9是本公开实施例的又一种电子设备的结构示意图。
具体实施方式
本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本公开的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
本公开实施例所涉及到的电子设备可以允许多个其他终端设备进行访问。上述电子设备包括终端设备。上述终端设备包括但不限于诸如具有触摸敏感表面(例如，触摸屏显示器和/或触摸板)的移动电话、膝上型计算机或平板计算机之类的便携式设备。还应当理解的是，在某些实施例中，所述终端设备并非便携式通信设备，而是具有触摸敏感表面(例如，触摸屏显示器和/或触摸板)的台式计算机。
深度学习的概念源于人工神经网络的研究。含多隐层的多层感知器就是一种深度学习结构。深度学习通过组合低层特征形成更加抽象的高层表示属性类别或特征，以发现数据的分布式特征表示。
深度学习是机器学习中一种基于对数据进行表征学习的方法。观测值(例如一幅图像)可以使用多种方式来表示，如每个像素点强度值的向量，或者更抽象地表示成一系列边、特定形状的区域等。而使用某些特定的表示方法更容易从实例中学习任务(例如，人脸识别或面部表情识别)。深度学习的好处是用非监督式或半监督式的特征学习和分层特征提取高效算法来替代手工获取特征。深度学习是机器学习研究中的一个新的领域，其动机在于建立、模拟人脑进行分析学习的神经网络，从而可以模仿人脑的机制来解释数据，例如图像、声音和文本。
同机器学习方法一样，深度机器学习方法也有监督学习与无监督学习之分。不同的学习框架下建立的学习模型很是不同。例如，卷积神经网络(Convolutional neural network，CNN)就是一种深度的监督学习下的机器学习模型，也可称为基于深度学习的网络结构模型，而深度置信网(Deep Belief Net，DBN)就是一种无监督学习下的机器学习模型。
下面对本公开实施例进行详细介绍。应理解，本公开实施例可以应用于对图像进行细胞核分割或者其他具有封闭结构的实例的分割，本公开实施例对此不做限定。
请参阅图1，图1是本公开实施例的一种图像处理方法的流程示意图。如图1所示，该图像处理方法包括如下步骤。
在步骤101、对第一图像进行处理，获得上述第一图像中多个像素点的预测结果。上述预测结果包括语义预测结果和中心相对位置预测结果。其中，上述语义预测结果指 示上述像素点位于实例区域或背景区域，上述中心相对位置预测结果指示上述像素点与实例中心之间的相对位置。
在101中，多个像素点可以为第一图像的所有或部分像素点，本公开实施例对此不做限定。上述第一图像可以包括通过各种图像采集设备(比如显微镜)获得的病理图像，比如细胞核图像等。本公开实施例对第一图像的获取方式以及实例的具体实现不做限定。
在本公开实施例中，可以通过多种方式对第一图像进行处理。例如，利用实例分割算法对第一图像进行处理，或者，可以将上述第一图像输入到神经网络进行处理，输出上述第一图像中多个像素点的预测结果，本公开实施例对此不做限定。
在一个例子中，可以通过基于深度学习的神经网络来获得上述第一图像中多个像素点的预测结果，比如深层融合网络(Deep Layer Aggregation，DLANet)，但本公开实施例对神经网络的具体实现不作限定。深层融合网络，也叫深层聚合网络，通过更深入的聚合来扩充标准体系结构，以更好地融合各层的信息。深层融合以迭代和分层方式合并特征层次结构，使网络具有更高的准确性和更少的参数。使用树型构造取代以往的线性构造，实现了对于网络的梯度回传长度的对数级别压缩，而不是线性压缩。这样，使得学习到的特征更具备描述能力，可以有效提高上述数值指标的预测精度。
可以对第一图像进行语义分割处理，得到第一图像中多个像素点的语义预测结果，并基于多个像素点的语义预测结果确定第一图像的实例分割结果。其中，语义分割处理用于将第一图像中的像素点按照语义含义的不同进行分组(Grouping)/分割(Segmentation)。例如，可以确定第一图像包含的多个像素点中每个像素点是实例还是背景，即位于实例区域还是位于背景区域。
像素点级别的语义分割可以对图像中的每个像素点划分出对应的类别，即实现像素点级别的分类；而类的具体对象，即为实例。实例分割不但要进行像素点级别的分类，还需在具体的类别基础上区别开不同的实例。比如说第一图像中有三个细胞核1、2、3，其语义分割结果都是细胞核，而实例分割结果却是不同的对象。
在本公开实施例中，可以对第一图像中的每一个像素点进行独立的实例判断，判断其所属的语义分割类别以及所属的实例ID。例如一张图像中有三个细胞核，则每个细胞核的语义分割类别都是1，但不同细胞核的ID分别是1、2、3，则可以通过上述细胞核ID来区分不同的细胞核。
像素点的语义预测结果可以指示上述像素点位于实例区域或背景区域。也就是说，像素点的语义预测结果指示该像素点为实例或者背景。
上述实例区域可以理解为实例所在的区域，背景区域为图像中除实例以外的其他区域。比如，假设第一图像为细胞图像，则像素点的语义预测结果可以包括用于指示像素点在细胞图像中为细胞核区域还是背景区域的指示信息。在本公开实施例中，可以通过多种方式指示像素点为实例区域还是背景区域。一些可能的实施方式中，像素点的语义预测结果可以为两个预设数值中的一个，这两个预设数值分别对应于实例区域和背景区域。例如，像素点的语义预测结果可以为0或正整数(例如1)。其中，0表示背景区域，正整数(例如1)表示实例区域，但本公开实施例不限于此。
上述语义预测结果可以是二值化结果。此时，可以对第一图像进行处理，得到多个像素点中每个像素点的实例区域预测概率，其中，实例区域预测概率指示该像素点位于实例区域的概率。然后，基于第二阈值对上述多个像素点中每个像素点的实例区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的语义预测结果。
在一个例子中，上述二值化处理的第二阈值可以为0.5。此时，将实例区域预测概率大于或等于0.5的像素点确定为位于实例区域的像素点，并将实例区域预测概率小于0.5的像素点确定为位于背景区域的像素点。相应地，可将实例区域预测概率大于或等于0.5的像素点的语义预测结果确定为1，并将实例区域预测概率小于0.5的像素点的语义预测结果确定为0，但本公开实施例不限于此。
像素点的预测结果可包括像素点的中心相对位置预测结果，用于指示上述像素点与该像素点所属实例中心之间的相对位置。在一个例子中，像素点的中心相对位置预测结果可以包括对像素点的中心向量的预测结果。例如，像素点的中心相对位置预测结果可表示为向量(x，y)，分别表示像素点的坐标与实例中心的坐标在横轴和纵轴上的差值。像素点的中心相对位置预测结果还可以通过其他方式实现，本公开实施例对此不做限定。
可以基于像素点的中心相对位置预测结果和该像素点的位置信息，确定像素点的实例中心预测位置，即像素点所属实例的中心的预测位置，并基于像素点的实例中心预测位置，确定像素点所属的实例，但本公开实施例对此不做限定。
在一个例子中，可以基于对第一图像的处理，确定第一图像中的至少一个实例中心的位置信息，并基于像素点的实例中心预测位置和至少一个实例中心的位置信息，确定像素点所属的实例。
在另一个例子中，可以将实例中心所属的一小块区域定义为实例中心区域。例如，实例中心区域是在该实例区域内并且小于该实例区域的区域，并且该实例中心区域的几何中心与该实例区域的几何中心重叠或邻近，例如，实例中心区域的中心为实例中心。该实例中心区域可以为圆形、椭圆或其他形状。上述实例中心区域可以根据需要进行设置，本公开实施例对实例中心区域的具体实现不做限制。
此时，可以确定第一图像中的至少一个实例中心区域，并基于像素点的实例中心预测位置与至少一个实例中心区域之间的位置关系，确定像素点所属的实例，但本公开实施例对其具体实现不做限定。
像素点的预测结果还可包括像素点的中心区域预测结果，指示像素点是否位于实例中心区域。相应地，可以基于多个像素点中每个像素点的中心区域预测结果，确定第一图像的至少一个实例中心区域。
在一个例子中，可以通过神经网络对第一图像进行处理，得到第一图像包含的多个像素点中每个像素点的中心区域预测结果。
上述神经网络可以是通过监督训练方式进行训练得到的。训练过程中利用的样本图像可以标注有实例信息，可以基于样本图像标注的实例信息确定实例的中心区域，并将确定的实例的中心区域作为监督来进行神经网络的训练。
可以基于实例信息，确定实例中心，并将包含实例中心的预设尺寸或面积的区域确定为实例的中心区域。还可以对样本图像进行腐蚀处理，得到腐蚀处理后的样本图像，并基于腐蚀处理后的样本图像确定实例的中心区域。
图像的腐蚀操作是表示用某种结构元素对图像进行探测，以便找出在图像内部可以放下该结构元素的区域。本公开实施例中提到的图像腐蚀处理可以包括上述腐蚀操作，腐蚀操作是结构元素在被腐蚀图像中平移填充的过程。从腐蚀后的结果来看，图像前景区域缩小，区域边界变模糊，同时一些比较小的孤立的前景区域被完全腐蚀掉，达到了滤波的效果。
比如，针对每一个实例蒙版，首先利用5×5的卷积核对实例蒙版(mask)进行图像 腐蚀处理。然后，将实例包括的多个像素点的坐标进行平均，得到实例的中心位置，并确定实例中的所有像素点到达该实例的中心位置的最大距离，并将与实例的中心位置之间的距离小于上述最大距离的30％的像素点确定为实例的中心区域的像素点，即得到实例的中心区域。这样，由样本图像中的实例蒙版缩小一圈后，进行图像二值化处理获得中心区域预测的二值图蒙版。
此外，可以基于样本图像中标注的实例中包含的像素点的坐标以及实例的中心位置，获得像素点的中心相对位置信息，即上述像素点与实例中心之间的相对位置信息，例如由像素点到实例中心的向量，并将该相对位置信息作为监督进行神经网络的训练，但本公开实施例不限于此。
在本公开实施例中，可以通过对第一图像进行处理，得到第一图像包含的多个像素点中每个像素点的中心区域预测结果。在一些可能的实现方式中，可以对上述第一图像进行处理，得到上述第一图像包含的多个像素点中每个像素点的中心区域预测概率；并基于第一阈值对上述多个像素点的中心区域预测概率进行二值化处理，得到上述多个像素点中每个像素点的中心区域预测结果。
其中，像素点的中心区域预测概率可以指像素点位于实例中心区域的概率。不位于实例中心区域的像素点可以是背景区域的像素点或者实例区域的像素点。
在本公开实施例中，二值化处理可以为固定阈值的二值化处理或者自适应阈值的二值化处理。例如双峰法、P参数法、迭代法和OTSU法等。本公开实施例对二值化处理的具体实现不做限定。上述二值化处理的第一阈值或第二阈值可以是预设的或者是根据实际情况确定的，本公开实施例对此不做限定。
可通过判断像素点的中心区域预测概率与上述第一阈值之间的大小关系，来获得像素点的中心区域预测结果。比如第一阈值可以为0.5。此时，可将中心区域预测概率大于或等于0.5的像素点确定为位于实例中心区域的像素点，并将中心区域预测概率小于0.5的像素点确定为不位于实例中心区域的像素点，从而得到每个像素点的中心区域预测结果。例如，将中心区域预测概率大于或等于0.5的像素点的中心区域预测结果确定为1，并将中心区域预测概率小于0.5的像素点的中心区域预测结果确定为0，但本公开实施例不限于此。
在获得上述预测结果之后可以执行步骤102。
在102、基于上述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定上述第一图像的实例分割结果。
在步骤101中，获得了上述语义预测结果和上述中心相对位置预测结果之后，可以确定位于实例区域的至少一个像素点以及上述至少一个像素点与其所属实例中心之间的相对位置信息。在一些可能的实现方式中，可以基于上述多个像素点中每个像素点的语义预测结果，从上述多个像素点中确定位于实例区域的至少一个第一像素点；基于第一像素点的中心相对位置预测结果，确定第一像素点所属的实例。
可以根据多个像素点中每个像素点的语义预测结果，确定出位于实例区域的至少一个第一像素点。具体地，将多个像素点中语义预测结果指示位于实例区域的像素点确定为第一像素点。
对于位于实例区域的像素点(即上述第一像素点)，可以根据像素点的中心相对位置预测结果，判断该像素点所属的实例。其中，第一图像的实例分割结果包括至少一个实例中每个实例包括的像素点，换句话说，包括位于实例区域的每个像素点所属的 实例。可以通过不同的实例标识或标号(例如实例ID)来区分不同的实例。其中，实例ID可以为大于0的整数。比如实例a的实例ID为1，实例b的实例ID为2，背景对应的实例ID为0。可以得到第一图像包含的多个像素点中每个像素点对应的实例标识，或者得到第一图像中每个第一像素点的实例标识，即位于背景区域的像素点不具有对应的实例标识，本公开实施例对此不做限定。
对于细胞实例分割中的像素点，若其语义预测结果为细胞且表示其中心相对位置预测结果的中心向量指向某个中心区域，则将此像素点分配给该细胞的细胞核区域(细胞核语义区域)。按照上述步骤对全部像素点进行分配，可以获得细胞分割结果。
在数字显微镜中进行细胞核分割可以提取细胞核的高质量形态学特征，也可以进行细胞核的计算病理学分析。这些信息是判断例如癌症级别、药物治疗有效性的重要依据。在过去人们常用大津算法(Otsu)和水线(也称分水岭或流域，watershed)阈值算法来解决细胞核实例分割的问题。但由于细胞核形态的多样性，上述方法效果不佳。实例分割可以依靠卷积神经网络(Convolutional Neural Network，CNN)，主要有基于MaskRCNN(Mask Regions with CNN features)和简单梳理全卷积网络(Fully Convolutional Network，FCN)的目标实例分割框架。但是，MaskRCN的缺点在于超参数繁多，对于具体问题要求人员具备很高的专业认知才能得到较好的结果，且该方法运行缓慢。FCN需要特殊的图像后处理才能把粘合的细胞分成多个实例，这也需要从业人员较高的专业知识。
本公开实施例中使用表示像素点相对于所属实例的中心的位置关系的中心向量来建模，使图像处理中的实例分割具备速度快、精度高的优点。对于细胞分割问题，上述FCN将部分实例收缩为边界类，然后使用针对性的后处理算法来修整边界所属实例的预测。相比之下，中心向量建模可以基于数据更精确的预测细胞核的边界状态，也无需复杂的专业后处理算法。上述MaskRCNN先通过矩形截取出每个独立实例的图像，再进行细胞、背景的二类预测。其中，由于细胞表现为聚集在一起的多个不规则类椭圆形，矩形截取后一个实例处于中心，别的实例仍然部分处于边缘，这不利于接下来的二类分割。相比之下，中心向量建模也不会有这类问题，而可以对于细胞核边界得出相对精确的预测，从而提高了整体预测精度。
本公开实施例可以应用于临床的辅助诊断中。医生在获得了病人的器官组织切片数字扫描图像后，可以将图像输入本公开实施例中的流程，得出每一个独立细胞核的像素点蒙版。然后，医生可以以此器官的每个独立细胞核的像素点蒙版为依据，计算该器官的细胞密度、细胞形态特征，进而得出更准确的医学判断。
本公开实施例通过基于第一图像包含的多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定上述第一图像的实例分割结果，可以使图像处理中的实例分割具备速度快、精度高的优点。
请参阅图2，图2是本公开实施例的另一种图像处理方法的流程示意图，图2是在图1的基础上进一步优化得到的。执行本公开实施例步骤的主体可以为前述的一种电子设备。如图2所示，该图像处理方法包括如下步骤：
在201、对第二图像进行预处理，得到第一图像，以使得上述第一图像满足预设对比度和/或预设灰度值。
本公开实施例中提到的第二图像可以为通过各种图像采集设备(比如显微镜)获得的多模态病理图像。上述多模态可以理解为图像类型可以是多样化的，并且图像大小、色彩、分辨率等特征可能不相同，呈现出的图像风格不一样，即上述第二图像可以 为一张或者多张。在病理切片的制作以及成像的过程中，由于组织类型、获取途径、成像设备等因素的不同，得到的病理影像数据通常差异很大。例如，不同显微镜下采集的病理图像，其分辨率会有很大的差异。通过光学显微镜可以获取病理组织的彩色图像(分辨率较低)，而电子显微镜通常只能采集到灰度图像(但分辨率较高)。然而，对于一套临床可用的病理系统，通常需要分析不同类型的、由不同成像设备获取的病理组织。
包含上述第二图像的数据集中，不同病人、不同器官、不同染色方法的图片复杂多样，因此可以首先通过步骤201降低第二图像的多样性。
执行本公开实施例步骤的主体可以为前述的一种电子设备。电子设备中可以存储有上述预设对比度和/或上述预设灰度值，可以将上述第二图像转换为满足上述预设对比度和/或上述预设灰度值的第一图像后，再执行步骤202。
本公开实施例中提到的对比度指的是一幅图像中明暗区域最亮的白和最暗的黑之间不同亮度层级的测量，差异范围越大代表对比越大，差异范围越小代表对比越小。
由于景物各点的颜色及亮度不同，摄成的黑白照片上或电视接收机重现的黑白图像上各点呈现不同程度的灰色。把白色与黑色之间按对数关系分成若干级，称为“灰度等级”。灰度等级的范围一般从0到255，白色为255，黑色为0。故黑白图片也称灰度图像，在医学、图像识别领域有很广泛的用途。
上述预处理还可以对上述第二图像的大小、分辨率、格式等参数进行统一。比如，可以对第二图像进行剪裁，获得预设图像尺寸的第一图像，比如统一为256*256尺寸的第一图像。电子设备还可以存储有预设图像大小和/或预设图像格式，在预处理时可以转换获得满足上述预设图像大小和/或预设图像格式的第一图像。
电子设备可以借助图像超分辨率(Image Super Resolution)以及图像转换等技术，将不同病理组织、不同成像设备获取的多模态病理图像进行统一，使它们可以作为本公开实施例中的图像处理流程的输入。此步骤也可以称为图像的归一化过程。转换为统一风格的图像，更便于后续对图像的统一处理。
图像超分辨率技术是指用图像处理的方法，通过软件算法(强调不变动成像硬件设备)的方式将已有的低分辨率(LR)图像转换成高分辨率(HR)图像的技术，可分为超分辨率复原和超分辨率图像重建(Super resolution image reconstruction，SRIR)。目前，图像超分辨率研究可分为三个主要范畴：基于插值、基于重建和基于学习的方法。超分辨率重建的核心思想就是用时间带宽(获取同一场景的多帧图像序列)换取空间分辨率，实现时间分辨率向空间分辨率的转换。通过上述预处理可以获得高分辨率的第一图像，对于医生做出正确的诊断是非常有帮助的。如果能够提供高分辨的图像，计算机视觉中的模式识别的性能也就会大大提高。
在202、对上述第一图像进行处理，获得上述第一图像中多个像素点的预测结果。上述预测结果包括语义预测结果、中心相对位置预测结果和中心区域预测结果。其中，上述语义预测结果指示上述像素点位于实例区域或背景区域，上述中心相对位置预测结果指示上述像素点与实例中心之间的相对位置，上述中心区域预测结果指示上述像素点是否位于实例中心区域。
其中，上述步骤202可以参考图1所示实施例的步骤101中的具体描述，此处不再赘述。
在203、基于上述多个像素点中每个像素点的语义预测结果，从上述多个像素点中确定位于实例区域的至少一个第一像素点。
基于上述多个像素点中每个像素点的语义预测结果，可以判断出每个像素点位于实例区域还是背景区域，从而可以从上述多个像素点中确定位于实例区域的至少一个第一像素点。
其中，实例区域可以参考图1所示实施例中的具体描述，此处不再赘述。
在204、基于上述多个像素点中每个像素点的中心区域预测结果，确定上述第一图像的至少一个实例中心区域。
其中，实例中心区域可以参考图1所示实施例中的具体描述，此处不再赘述。
其中，中心相对位置预测结果可以参考图1所示实施例中的具体描述，此处不再赘述。
本公开实施例中，中心区域预测结果可以指示像素点是否位于实例中心区域，由此可以通过参考中心区域预测结果，确定位于实例中心区域的像素点。而位于实例中心区域的这些像素点可以组成实例中心区域，由此可以确定出至少一个实例中心区域。
可以基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
其中，连通区域(Connected Component)一般是指图像中具有相同像素点值且位置相邻的前景像素点组成的图像区域(Region，Blob)。上述连通域搜索可以理解为连通区域分析(Connected Component Analysis，Connected Component Labeling)，用于将图像中的各个连通区域找出并标记。
连通区域分析是在国际计算机视觉与模式识别会议(Conference on Computer Vision and Pattern Recognition，CVPR)和图像分析处理的众多应用领域中较为常用和基本的方法。例如：光学字符识别(Optical Character Recognition，OCR)中字符分割提取(车牌识别、文本识别、字幕识别等)、视觉跟踪中的运动前景目标分割与提取(行人入侵检测、遗留物体检测、基于视觉的车辆检测与跟踪等)、医学图像处理(感兴趣目标区域提取)等等。也就是说，在需要将前景目标提取出来以便后续进行处理的任意应用场景中都能够用到连通区域分析方法，通常连通区域分析处理的对象是一张二值化后的图像(二值图像)。
对于集合S存在一条通路的条件是，通路的像素点的某种排列使得相邻像素点满足某种邻接关系。例如，假设点p到点q之间有A1,A2,A3.....An个像素点，且相邻像素点都满足某种邻接，则p和q间存在通路。如果通路首尾相连，则称闭合通路。S集合中的一点p只存在一条通路，则称为一个连通分量，如果S只有一个连通分量，则称为一个连通集。
对于R为一个图像子集，如果R连通的，则称R为一个区域。对于所有不连接的K个区域，其并集Rk构成了图像的前景，Rk的补集称为背景。
基于上述每个像素点的中心区域预测结果，对上述第一图像进行连通域搜索处理，可以得到至少一个实例中心区域，再执行步骤205。
具体的，对于二值化处理后的第一图像，可以找中心区域为1的连通域，以确定实例中心区域，为每个连通域分配一个独立ID。
对于细胞分割，可以基于细胞核中的像素点的坐标和表示该像素点相对于所属实例的中心的位置关系的中心向量，确定上述中心向量的指向位置是否处于上述中心区域。若像素点的中心向量的指向位置处于中心区域，则为该像素点分配细胞核的ID； 否则，表示该像素点不属于任意一个细胞核，可以就近分配。
可以使用随机游走算法对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
随机游走(random walk)也称随机漫步、随机行走等，是指基于过去的表现，无法预测将来的发展步骤和方向。随机游走的核心概念是指，任何无规则行走者所带的守恒量对应着一个扩散运输定律，接近于布朗运动，是布朗运动理想的数学状态。本公开实施例中针对图像处理的随机游走(random walk)的基本思想是，将图像看成由固定的顶点和边组成的连通带权无向图，从未标记顶点开始随机漫步，首次到达各类标记顶点的概率代表了未标记点归属于标记类的可能性，把最大的概率所在类的标签赋给未标记顶点，完成分割。通过上述随机游走算法可以实现对不属于任意一个中心区域的像素点的分配，以获得上述至少一个实例中心区域。
可以通过深度层级融合网络模型输出像素点连接图，在连通域搜索处理后可以得出实例分割结果。可以在上述实例分割结果中对每个实例区域赋予随机色彩以便于可视化。
其中，上述步骤203和步骤204也可以不分先后次序执行；在确定上述至少一个实例中心区域之后，可以执行步骤205。
在205、基于每个第一像素点的中心相对位置预测结果，从上述至少一个实例中心区域中确定上述每个第一像素点对应的实例中心区域。
具体的，可以基于上述第一像素点的位置信息和上述第一像素点的中心相对位置预测结果，确定上述第一像素点的中心预测位置。
在步骤202中可以获得像素点的位置信息、具体可以为像素点的坐标。而根据上述第一像素点的坐标和上述第一像素点的中心相对位置预测结果，可以确定上述第一像素点的中心预测位置。上述中心预测位置可以指示预测的上述第一像素点所属的实例中心区域的中心位置。
基于第一像素点的中心预测位置和至少一个实例中心区域的位置信息，可以从上述至少一个实例中心区域中确定上述第一像素点对应的实例中心区域。
在步骤204中，可以获得实例中心区域的位置信息，也可以由坐标表示。进而，可以基于第一像素点的中心预测位置和至少一个实例中心区域的位置信息，判断上述第一像素点的中心预测位置是否属于上述至少一个实例中心区域，以此从上述至少一个实例中心区域中确定第一像素点对应的实例中心区域。
具体的，可以响应于第一像素点的中心预测位置属于至少一个实例中心区域中的第一实例中心区域，将上述第一实例中心区域确定为上述第一像素点对应的实例中心区域，并可以将该像素点分配给该实例中心区域。
响应于第一像素点的中心预测位置不属于至少一个实例中心区域中的任意实例中心区域，进行就近分配，即将至少一个实例中心区域中与第一像素点的中心预测位置距离最近的实例中心区域确定为该第一像素点对应的实例中心区域。
本公开实施例在上述步骤202的输出可以有三个分支：第一个是语义判断分支，包含2个通道，以输出每个像素点位于实例区域或者背景区域；第二个是中心区域分支，包含2个通道，以输出每个像素点位于中心区域或者非中心区域；第三个是中心向量分支，包括2个通道，以输出每个像素点与实例中心之间的相对位置，具体可以包含像素点指向其所属实例的几何中心的向量横纵分量。
在本公开实施例中，所述实例为第一图像中的分割对象，具体可以为第一图像中的封闭性结构。例如，分割对象可以为细胞核。这样，由于上述中心区域为一个细胞核的中心区域，在确定上述中心区域后，实际初步确定了细胞核的位置，可以为每个细胞核分配数字编号，即上述实例ID。
具体的，设输入的第二图片为[高，宽，3]的3通道图片，本公开实施例在步骤202可以得到三个[高，宽，2]的数组，依次为每个像素点的语义预测概率、中心区域预测概率和中心相对位置预测结果。然后，可以对上述中心区域预测概率进行阈值为0.5的二值化，再通过连通域搜索处理得到每个细胞核的中心区域，并且赋予其独立的数字编号，上述每个细胞分配的数字编号即为前述实例ID，以便于区分不同细胞核。
比如，假设在步骤203中已经确定一个像素点a的语义预测结果为细胞核而非背景(确定其属于细胞核语义区域)，并在步骤202中已经获得了该像素点a的中心向量，若该像素点a的中心向量指向在步骤204中获得的至少一个实例中心区域中的第一中心区域，则说明该像素点a与该第一中心区域有对应关系。具体表现为，该像素点a属于该第一中心区域所在的细胞核A，第一中心区域为该细胞核A的中心区域。
以细胞分割为例，通过上述步骤，可以分割出细胞核与图像背景，可以对全部属于细胞核的像素点进行分配，确定每个像素点所属的细胞核、所属的细胞核中心区域或者所属的细胞核的中心，实现对细胞进行更精准的分割，获得精确的实例分割结果。
本公开实施例中使用中心向量来建模，可以对于细胞核边界得出精确的预测，从而提高了整体预测精度。
使用本公开实施例中的中心向量方法，不仅运行速度快，可以达到每秒3图的处理量，而且无需从业人员较高的领域知识，就能在任意实例分割问题中获取一定标注数据后处理取得较好的结果。
本公开实施例可以应用于临床的辅助诊断中，具体描述可参考图1所示实施例，此处不再赘述。
本公开实施例通过对第二图像进行预处理得到第一图像，并基于第一图像包含的多个像素点中每个像素点的语义预测结果、中心区域预测结果、中心相对位置预测结果，确定上述第一图像中位于实例区域的每个第一像素点对应的实例中心区域，可以有效实现对实例的精准分割，可以使图像处理中的实例分割具备速度快、精度高的优点。
请参阅图3，图3是本公开实施例的一种细胞实例分割结果示意图。如图所示，以细胞实例分割为例，使用本公开实施例中的方法进行处理，同时具备速度快、精度高的特点。结合图3可以便于更清楚地理解图1和图2所述实施例中的方法。通过深度层级融合网络模型可以获得更准确的预测指标，并可使用已有数据集对预测指标进行标注。前述实施例中的语义预测结果、中心区域预测结果和中心相对位置预测结果，体现在图3中分别包括对像素点A、像素点B、像素点C和像素点D的语义标注、中心标注和中心向量标注。如图所示，一个细胞核可包括细胞核语义区域和细胞核中心区域。针对图中像素点，若像素点的语义标注为1，说明该像素点属于细胞核，为0则为图像背景；若像素点的中心标注为1则说明该像素点为细胞核区域的中心，此时该像素点的中心向量标注为(0,0)，可作为其他像素点的参考(比如图中的像素点A和像素点D，像素点A的确定也可以代表一个细胞核的确定)。每个像素点都对应一个坐标，而中心向量标注则是像素点相对于细胞核中心的像素点的坐标，比如像素点B相对于像素点A的中心向量标注为(-5，-5)，而属于中心的像素点的中心向量标注则为(0,0)，比如像素点A和像素点D。在本公开实施例中可以判断出上述像素点B属于上述像素点A所 属的细胞核区域，即将像素点B分配给像素点A所属的细胞核区域，但不在该细胞核中心区域内而是在上述细胞核语义区域内。类似地完成全部分割过程，可获得相对精确的细胞实例分割结果。
上述主要从方法侧执行过程的角度对本公开实施例的方案进行了介绍。可以理解的是，电子设备为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所的实施例描述的各示例的单元及算法步骤，本公开能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对特定的应用使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
本公开实施例可以根据上述方法示例对电子设备进行功能单元的划分，例如，可以对应各个功能划分各个功能单元，也可以将两个或两个以上的功能集成在一个处理单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。需要说明的是，本公开实施例中对单元的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
请参阅图4，图4是本公开实施例的一种电子设备的结构示意图。如图4所示，该电子设备400包括预测模块410和分割模块420，其中：所述预测模块410，用于对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实例中心之间的相对位置；所述分割模块420，用于基于所述多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定所述第一图像的实例分割结果。
所述电子设备400还可包括预处理模块430，用于对第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设对比度和/或预设灰度值。
所述分割模块420可包括第一单元421和第二单元422，其中：所述第一单元421，用于基于所述多个像素点中每个像素点的语义预测结果，从所述多个像素点中确定位于实例区域的至少一个第一像素点；所述第二单元422，用于基于每个第一像素点的中心相对位置预测结果，确定所述每个第一像素点所属的实例。
所述预测结果还可包括中心区域预测结果，所述中心区域预测结果指示所述像素点是否位于实例中心区域。在此情况下，所述分割模块420还包括第三单元423，用于基于所述多个像素点中每个像素点的中心区域预测结果，确定所述第一图像的至少一个实例中心区域；所述第二单元422具体用于，基于每个第一像素点的中心相对位置预测结果，确定所述每个第一像素点对应的实例中心区域。
所述第三单元423具体可用于，基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
所述第二单元422具体可用于：基于所述第一像素点的位置信息和所述第一像素点的中心相对位置预测结果，确定所述第一像素点的中心预测位置；基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
所述第二单元422具体可用于：响应于所述第一像素点的中心预测位置属于所述至少一个实例中心区域中的第一实例中心区域，将所述第一实例中心区域确定为所述第一像素点对应的实例中心区域。
所述第二单元422具体可用于：响应于所述第一像素点的中心预测位置不属于所述至少一个实例中心区域中的任意实例中心区域，将所述至少一个实例中心区域中与所述第一像素点的中心预测位置距离最近的实例中心区域确定为所述第一像素点对应的实例中心区域。
所述预测模块410可包括概率预测单元411和判断单元412，其中：所述概率预测单元411，用于对所述第一图像进行处理，得到所述第一图像中多个像素点各自的中心区域预测概率；所述判断单元412，用于基于第一阈值对所述多个像素点各自的中心区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的中心区域预测结果。
所述预测模块410具体可用于，将第一图像输入到神经网络进行处理，输出所述第一图像中多个像素点的预测结果。
本公开实施例中使用中心向量来建模，可以对于细胞核边界得出精确的预测，从而提高了整体预测精度。
使用本公开实施例中的电子设备400，可以实现前述图1和图2实施例中的图像处理方法，通过中心向量方法进行实例分割，不仅运行速度快，可以达到每秒3图的处理量，而且无需从业人员较高的领域知识，就能在任意实例分割问题中获取一定标注数据后处理取得较好的结果。
实施图4所示的电子设备400，电子设备400可以通过基于第一图像包含的多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定上述第一图像的实例分割结果，可以使图像处理中的实例分割具备速度快、精度高的优点。
请参阅图5，图5是本公开实施例的一种图像处理方法的流程示意图。该方法可以由任意电子设备执行，例如终端设备、服务器或者处理平台等，本公开实施例对此不做限定。如图5所示，该图像处理包括如下步骤。
在501、获取N组实例分割输出数据。其中，上述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且上述N组实例分割输出数据具有不同的数据结构，上述N为大于1的整数。
首先，图像处理中的实例分割问题定义为：对于一张输入图像，要对每一个像素点进行独立的判断，判断其所属语义类别以及实例ID。例如图像中有三个细胞核1、2、3，其语义类别都是细胞核，而实例分割结果却是不同的对象。
实例分割可以参考图1所示实施例的具体描述，并在此不再赘述。
实例分割也可以依靠实例分割算法来实现，例如基于支持向量机的实例分割算法等机器学习模型，本公开实施例对实例分割模型的具体实现不作限定。
不同的实例分割模型各有其优势与缺点，本公开实施例通过集成多个实例分割模型来整合不同单模型的优点。
在执行步骤501之前，可以使用不同的实例分割模型对图像分别进行处理，比如使用MaskRCNN和FCN分别对图像进行处理，获得实例分割输出结果。假设有N个实例分割模型，可以获取N个实例分割模型中每个实例分割模型的实例分割结果(以下称为实例分割输出数据)，即获得N组实例分割输出数据。或者，可以从其他设备处获取该N组实例分割输出数据，本公开实施例对获取N组实例分割输出数据的方式不作限定。
在使用实例分割模型对图像进行处理之前，还可以对图像进行预处理，例如对 比度和/或灰度调整，或者裁剪、水平和垂直翻转、旋转、缩放、噪声去除等一种或任意多项操作，以使得预处理后的图像满足实例分割模型对于输入图像的要求，本公开实施例对此不做限定。
在本公开实施例中，N个实例分割模型输出的实例分割输出数据可以具有不同的数据结构或含义。举例来讲，对于一个维度为[高，宽，3]的图像的输入，实例分割输出数据包括[高，宽]的数据。其中，实例ID为0表示背景，大于0的不同数字表示不同的实例。假设有3个实例分割模型，不同的实例分割模型对应不同的算法或者神经网络结构，其中，第1个实例分割模型的实例分割输出数据是[边界、目标、背景]的三分类概率图；第2个实例分割模型的实例分割输出数据是[边界、背景]的二分类概率图和维度为[目标、背景]的二分类图；第3个实例分割模型的实例分割输出数据是[中心区域、目标整体、背景]的三分类概率图，等等。不同的实例分割模型拥有不同意义的数据输出。此时，无法使用任意加权平均算法来整合各个实例分割模型的输出以取得更稳定、更高精度的结果。本公开实施例中的方法可以在此N组具有不同数据结构的实例分割输出数据的基础上进行跨实例分割模型的集成。
在获取上述N组实例分割输出数据之后，可以执行步骤502。
在502、基于上述N组实例分割输出数据，得到上述图像的集成语义数据和集成中心区域数据。其中，上述集成语义数据指示上述图像中位于实例区域的像素点，上述集成中心区域数据指示上述图像中位于实例中心区域的像素点。
具体的，电子设备可以将上述N组实例分割输出数据进行转换处理，获得图像的集成语义数据和集成中心区域数据。
本公开实施例中提到的语义分割是计算机视觉中的基本任务，可参考图1所示实施例中的具体描述，并在此不再赘述。
像素级别的语义分割可以参考图1所示实施例中的具体描述，并在此不再赘述。
上述实例区域可以理解为图像中的实例所在的区域，即除去背景区域以外的区域，上述集成语义数据则可以指示上述图像中位于实例区域的像素点。比如，针对细胞核分割的处理，上述集成语义数据可以包括位于细胞核区域的像素点的判断结果。
而上述集成中心区域数据可以指示上述图像中位于实例中心区域的像素点。
实例中心区域可以参考图1所示实施例中的具体描述，并在此不再赘述。
具体的，可以先基于N个实例分割模型中每个实例分割模型的实例分割输出数据，得到每个实例分割模型的语义数据和中心区域数据，即一共N组语义数据和N组中心区域数据。然后，基于上述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据进行集成处理，得到上述图像的集成语义数据和集成中心区域数据。
对于N个实例分割模型中每个实例分割模型，可以确定在该实例分割模型中的每个像素点对应的实例标识信息(实例ID)，再基于在该实例分割模型中每个像素点对应的实例标识信息，得到每个像素点在该实例分割模型中的语义预测值。其中，实例分割模型的语义数据包括上述图像的多个像素点中每个像素点的语义预测值。
二值化(Thresholding)是图像分割的一种简单的方法。二值化可以把灰度图像转换成二值图像。例如，可以把大于某个临界灰度值的像素点灰度设为灰度极大值，把小于这个值的像素点灰度设为灰度极小值，从而实现二值化。
二值化处理可以参考图1所示实施例中的具体描述，并在此不再赘述。
在本公开实施例中，可以通过对第一图像进行处理，得到第一图像包含的多个像素点中每个像素点的语义预测结果。可通过判断像素点的语义预测值与上述第一阈值之间的大小关系，来获得像素点的语义预测结果。上述第一阈值可以是预设的或者是根据实际情况确定的，本公开实施例对此不做限定。
在得到上述图像的集成语义数据和集成中心区域数据之后，可以执行步骤503。
503、基于上述图像的集成语义数据和集成中心区域数据，获得上述图像的实例分割结果。
可以基于上述图像的集成中心区域数据，得到上述图像的至少一个实例中心区域。然后，可以基于上述至少一个实例中心区域和上述图像的集成语义数据，确定上述图像的多个像素点中每个像素点所属的实例。
上述集成语义数据指示图像中位于实例区域的至少一个像素点。例如，集成语义数据可以包括图像的多个像素点中每个像素点的集成语义值，集成语义值用于指示像素点是否位于实例区域，或用于指示像素点位于实例区域或背景区域。上述集成中心区域数据指示上述图像中位于实例中心区域的至少一个像素点。例如，集成中心区域数据包括图像的多个像素点中每个像素点的集成中心区域预测值，集成中心区域预测值用于指示像素点是否位于实例中心区域。
通过上述集成语义数据可以确定图像的实例区域中包含的至少一个像素点，通过上述集成中心区域数据可以确定图像的实例中心区域中包含的至少一个像素点。基于上述图像的集成中心区域数据和集成语义数据，则可以确定上述图像的多个像素点中每个像素点所属的实例，并获得图像的实例分割结果。
通过上述方法获得的实例分割结果集成了N个实例分割模型的实例分割输出结果，整合了不同实例分割模型的优点，不再要求不同实例分割模型拥有相同含义的数据输出，并且提高了实例分割精度。
本公开实施例通过基于通过N个实例分割模型对图像进行处理获得的N组实例分割输出数据，得到上述图像的集成语义数据和集成中心区域数据，进而基于上述图像的集成语义数据和集成中心区域数据，获得上述图像的实例分割结果，可以实现各个实例分割模型的优势互补，而不再要求各个模型具有相同结构或含义的数据输出，在实例分割问题中取得更高的精度。
请参阅图6，图6是本公开实施例的另一种图像处理方法的流程示意图，图6是在图5的基础上进一步优化得到的。该方法可以由任意电子设备执行，例如终端设备、服务器或者处理平台等，本公开实施例对此不做限定。如图6所示，该图像处理方法包括如下步骤：
在601、获取N组实例分割输出数据。其中，上述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且上述N组实例分割输出数据具有不同的数据结构，上述N为大于1的整数。
其中，上述步骤601可以参考图5所示实施例的步骤501中的具体描述，此处不再赘述。
602、基于上述实例分割模型的实例分割输出数据，确定在上述实例分割模型中，上述图像中位于实例区域的至少两个像素点。
实例中心区域可以参考图1所示实施例中的具体描述，并在此不再赘述。实例分割输出数据可以包括图像中位于实例区域的至少两个像素点中每个像素点对应的实 例标识信息，例如，实例ID为1、2或3等大于0的整数，或者也可以为其他数值。位于背景区域的像素点对应的实例标识信息可以为预设值，或者位于背景区域的像素点不对应任何实例标识信息。这样可以基于实例分割输出数据中多个像素点中每个像素点对应的实例标识信息，确定图像中位于实例区域的至少两个像素点。
实例分割输出数据也可以不包括每个像素点对应的实例标识信息。此时，可以通过对实例分割输出数据进行处理，得到图像中位于实例区域的至少两个像素点，本公开实施例对此不做限定。
在确定上述图像中位于实例区域的至少两个像素点之后，可以执行步骤603。
603、基于上述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定上述实例分割模型的实例中心位置。
在确定了上述实例分割模型中位于实例区域的至少两个像素点之后，可以获得上述至少两个像素点的位置信息。其中，该位置信息可以包括像素点在图像中的坐标，但本公开实施例不限于此。
可以根据上述至少两个像素点的位置信息，确定上述实例分割模型的实例中心位置。上述实例中心位置不局限于为该实例的几何中心位置，而可为预测的该实例区域的中心位置，即可以理解为上述实例中心区域中的任一位置。
可以将上述位于实例区域的至少两个像素点的位置的平均值，作为上述实例分割模型的实例中心位置。
具体的，可以将上述位于实例区域的至少两个像素点的坐标取平均值，作为上述实例分割模型的实例中心位置的坐标，以确定上述实例中心位置。
604、基于上述实例分割模型的实例中心位置和上述至少两个像素点的位置信息，确定上述实例分割模型的实例中心区域。
具体的，可以基于上述实例分割模型的实例中心位置和上述至少两个像素点的位置信息，确定上述至少两个像素点与上述实例中心位置的最大距离，再基于上述最大距离确定第一阈值。然后，可以将上述至少两个像素点中与上述实例中心位置之间的距离小于或等于上述第一阈值的像素点，确定为实例中心区域的像素点。
比如，可以基于上述实例分割模型的实例中心位置和上述至少两个像素点的位置信息，计算每一个像素点到达该实例中心位置的距离(像素点距离)。电子设备中可以预先设置上述第一阈值的算法，比如上述第一阈值可以设置为上述像素点距离中最大距离的30％。在确定上述像素点距离中最大距离之后，可以计算获得上述第一阈值。以此为基础，保留像素点距离小于上述第一阈值的像素点，确定这些像素点为上述实例中心区域的像素点，即确定了上述实例中心区域。
还可以对样本图像进行腐蚀处理。腐蚀处理可以参考图1所示实施例中的具体描述，并在此不再赘述。
此外，像素点的中心相对位置信息可以参考图1所示实施例中的具体描述，并在此不再赘述。
605、基于上述N个实例分割模型中每个实例分割模型的语义数据，确定上述图像的多个像素点中每个像素点的语义投票值。
电子设备可以基于上述N个实例分割模型中每个实例分割模型的语义数据，对多个像素点中每个像素点进行语义投票，确定上述图像的多个像素点中每个像素点的语 义投票值。比如，可使用基于滑动窗口的投票对上述实例分割模型的语义数据进行处理，确定上述每个像素点的语义投票值，进而可以执行步骤606。
606、对上述多个像素点中每个像素点的语义投票值进行二值化处理，得到上述图像中每个像素点的集成语义值。其中，上述图像的集成语义数据包括上述多个像素点中每个像素点的集成语义值。
可以对每个像素点的来自上述N个实例分割模型的语义投票值进行二值化处理，得到上述图像中每个像素点的集成语义值。可以理解为，不同实例分割模型得到的语义蒙版相加得到集成语义蒙版。
具体的，可以基于上述多个实例分割模型的个数N，确定第二阈值；基于上述第二阈值，对上述多个像素点中每个像素点的语义投票值进行二值化处理，得到上述图像中每个像素点的集成语义值。
由于上述多个像素点中每个像素点的集成语义值有可能取值为实例分割模型个数，可以基于上述多个实例分割模型的个数N，确定第二阈值。比如，上述第二阈值可以为N/2的向上取整结果。
可以以第二阈值为该步骤中二值化处理的判断依据，得到上述图像中每个像素点的集成语义值。电子设备中可以存储有上述第二阈值的计算方法，比如规定上述预设像素点阈值为N/2，若N/2不为整数则向上取整。比如，4个实例分割模型获得的4组实例分割输出数据，则N＝4，4/2＝2，此时的第二阈值是2。相应地，比较上述语义投票值和上述第二阈值，语义投票值大于等于2的截断为1，小于2的截断为0，由此得到上述图像中每个像素点的集成语义值，此时输出的数据具体可以为集成语义二值图。上述集成语义值可以理解为上述每个像素点的语义分割结果，可以以此为基础确定该像素点所属的实例，实现实例分割。
607、基于上述图像的多个像素点中每个像素点的集成语义值和上述至少一个实例中心区域，进行随机游走，得到上述每个像素点所属的实例。
随机游走可以参考图1所示实施例中的具体描述，并在此不再赘述。
基于上述图像的多个像素点中每个像素点的集成语义值和上述至少一个实例中心区域，使用随机游走的形式来根据像素点的集成语义值判断像素点的分配情况，从而得到上述每个像素点所属的实例。比如，可以将离像素点最近的实例中心区域对应的实例确定为该像素点所属的实例。本公开实施例可以通过得到最终的集成语义图和集成中心区域图，结合上述连通区域搜索和随机游走的一种具体实现(就近分配)确定实例的像素点分配，获得最后的实例分割结果。
通过上述方法获得的实例分割结果集成了N个实例分割模型的实例分割输出结果，整合了这些实例分割模型的优点，不再要求不同实例分割模型拥有相同含义的连续概率图输出，并且提高了实例分割精度。
本公开实施例中的方法，适用于任意实例分割问题中。例如，可应用在临床的辅助诊断中，并可以参考图1所示实施例中的具体描述，并在此不再赘述。又如在蜂巢四周，饲养员获得了蜂巢四周密集的蜜蜂飞舞图像后，可以使用本算法获得每一只独立蜜蜂的实例像素点蒙版，可进行宏观的蜜蜂计数、行为模式计算等，具有很大的实用价值。
本公开实施例的具体应用中，对于自底向上的方法，可以优选应用UNet模型。UNet首先被开发用于语义分割，并有效地从多个尺度融合信息。对于自顶向下的方法， 可以应用MaskR-CNN模型。MaskR-CNN通过为分割任务添加头部来扩展更快的R-CNN。此外，所提出的MaskR-CNN中可以将跟踪特征与输入对齐，避免了双线性插值的任何量化。对齐对于像素点级任务，比如实例分割任务是十分重要的。
UNet模型的网络结构由收缩路径(contracting path)和扩张路径(expanding path)组成。其中，收缩路径用于获取上下文信息(context)，扩张路径用于精确的定位(localization)，且两条路径相互对称。该网络能够从极少图像端对端进行训练，并且对于分割电子显微镜中的神经元等细胞结构的表现好于以前最好的方法(滑动窗口卷积网络)。除此之外运行速度也非常快，
可以利用UNet和Mask R-CNN模型对实例进行分割预测，得到每个实例分割模型的语义蒙版，并通过像素点投票(Vote)进行集成。然后通过腐蚀处理来计算每个实例分割模型的中心蒙版，并对中心蒙版进行集成。最后，利用随机游走算法从集成的语义蒙版和中心蒙版中获得实例分割结果。
针对上述结果可以采用交叉验证(Cross-validation)方法进行评估。交叉验证主要用于建模应用中。在给定的建模样本中，拿出大部分样本进行建模型，留小部分样本用刚建立的模型进行预报，并求这小部分样本的预报误差，记录它们的平方加和。本公开实施例可采用3倍交叉验证进行评估，将三个AJI(5)得分0.605、0.599、0.589的UNet模型与一个AJI(5)得分0.565的MaskR-CNN模型结合，使用本公开实施例的方法获得的结果最后AJI(5)得分为0.616，可见本公开的图像处理方法具有明显的优势。
本公开实施例通过基于利用N个实例分割模型对图像进行处理获得的实例分割输出数据，确定上述实例分割模型的实例中心区域，并基于上述图像的多个像素点中每个像素点的集成语义值和上述至少一个实例中心区域进行随机游走，得到上述每个像素点所属的实例，可以实现各个实例分割模型的优势互补，不再要求各个模型具有相同结构或含义的数据输出，在实例分割问题中取得更高的精度。
请参阅图7，图7是本公开实施例的一种细胞实例分割的图像表现形式示意图。如图所示，以细胞实例分割为例，使用本公开实施例中的方法进行处理，可以获得精度更高的实例分割结果。使用N种实例分割模型(图中仅展示4种)分别给出输入图片的实例预测蒙版(图中不同色彩表示不同的细胞实例)，将实例预测蒙版转换为使用语义预测分割的语义蒙版和使用中心预测分割的中心区域蒙版后，分别进行像素点投票，再进行集成，最终获得实例分割结果。可以看出，在该过程中修复了方法1的右侧三细胞漏检两个的错误，修复了方法2的中间两细胞粘合的错误，还修复了4个方法都没能发现的左下角其实是三个细胞，中间还有个小细胞存在的现象。该集成方法可以在任意实例分割模型上集成，整合了不同方法的优点。通过上述举例可以更加清晰地了解前述实施例的具体过程及其优势。
上述主要从方法侧执行过程的角度对本公开实施例的方案进行了介绍。可以理解的是，电子设备为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本公开能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对特定的应用使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
本公开实施例可以根据上述方法示例对电子设备进行功能单元的划分，例如，可以对应各个功能划分各个功能单元，也可以将两个或两个以上的功能集成在一个处理 单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。需要说明的是，本公开实施例中对单元的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
请参阅图8，图8是本公开实施例的一种电子设备的结构示意图。如图8所示，该电子设备800包括：获取模块810、转换模块820和分割模块830，其中：所述获取模块810，用于获取N组实例分割输出数据，其中，所述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且所述N组实例分割输出数据具有不同的数据结构，所述N为大于1的整数；所述转换模块820，用于基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，其中，所述集成语义数据指示所述图像中位于实例区域的像素点，所述集成中心区域数据指示所述图像中位于实例中心区域的像素点；所述分割模块830，用于基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果。
所述转换模块820可包括第一转换单元821和第二转换单元822，其中：所述第一转换单元821，用于基于所述N个实例分割模型中每个实例分割模型的实例分割输出数据，得到所述每个实例分割模型的语义数据和中心区域数据；所述第二转换单元822，用于基于所述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据，得到所述图像的集成语义数据和集成中心区域数据。
所述第一转换单元821具体可用于：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中所述图像的多个像素点中每个像素点对应的实例标识信息；基于所述实例分割模型中所述多个像素点中每个像素点对应的实例标识信息，得到所述每个像素点在所述实例分割模型中的语义预测值，其中，所述实例分割模型的语义数据包括所述图像的多个像素点中每个像素点的语义预测值。
所述第一转换单元821具体还可用于：基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点；基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定所述实例分割模型的实例中心位置；基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域。
所述转换模块820还可包括腐蚀处理单元823，用于对所述实例分割模型的实例分割输出数据进行腐蚀处理，得到实例分割模型的腐蚀数据；所述第一转换单元821具体可用于，基于所述实例分割模型的腐蚀数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点。
所述第一转换单元821具体可用于，将所述位于实例区域的至少两个像素点的位置的平均值，作为所述实例分割模型的实例中心位置。
所述第一转换单元821具体还可用于：基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述至少两个像素点与所述实例中心位置的最大距离；基于所述最大距离，确定第一阈值；将所述至少两个像素点中与所述实例中心位置之间的距离小于或等于所述第一阈值的像素点确定为实例中心区域的像素点。
所述转换模块820具体可用于：基于所述N个实例分割模型中每个实例分割模型的语义数据，确定所述图像的多个像素点中每个像素点的语义投票值；对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值，其中，所述图像的集成语义数据包括所述多个像素点中每个像素点的集成语义值。
所述转换模块820，具体还可用于：基于所述多个实例分割模型的个数N，确定 第二阈值；基于所述第二阈值，对所述多个像素点中每个像素点的语义投票值进行二值化处理，得到所述图像中每个像素点的集成语义值。
所述第二阈值可为N/2的向上取整结果。
所述分割模块830，可包括中心区域单元831和确定单元832，其中：所述中心区域单元831，用于基于所述图像的集成中心区域数据，得到所述图像的至少一个实例中心区域；所述确定单元832，用于基于所述至少一个实例中心区域和所述图像的集成语义数据，确定所述图像的多个像素点中每个像素点所属的实例。
所述确定单元832，具体可用于基于所述图像的多个像素点中每个像素点的集成语义值和所述至少一个实例中心区域，进行随机游走，得到所述每个像素点所属的实例。
实施图8所示的电子设备800，电子设备800可以基于通过N个实例分割模型对图像进行处理获得的N组实例分割输出数据，得到上述图像的集成语义数据和集成中心区域数据，进而基于上述图像的集成语义数据和集成中心区域数据，获得上述图像的实例分割结果，可以实现各个实例分割模型的优势互补，而不再要求各个模型具有相同结构或含义的数据输出，在实例分割问题中取得更高的精度。
请参阅图9，图9是本公开实施例的另一种电子设备的结构示意图。如图9所示，该电子设备900包括处理器901和存储器902。其中，电子设备900还可以包括总线903，处理器901和存储器902可以通过总线903相互连接，总线903可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。总线903可以分为地址总线、数据总线、控制总线等。为便于表示，图9中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。其中，电子设备900还可以包括输入输出设备904，输入输出设备904可以包括显示屏，例如液晶显示屏。存储器902用于存储计算机程序；处理器901用于调用存储在存储器902中的计算机程序执行上述图1、图2、图5和图6实施例中提到的部分或全部方法步骤。
实施图9所示的电子设备900，电子设备900可以通过基于第一图像包含的多个像素点中每个像素点的语义预测结果和中心相对位置预测结果，确定上述第一图像的实例分割结果，可以使图像处理中的实例分割具备速度快、精度高的优点。
实施图9所示的电子设备900，电子设备900可以基于通过N个实例分割模型对图像进行处理获得的N组实例分割输出数据，得到上述图像的集成语义数据和集成中心区域数据，进而基于上述图像的集成语义数据和集成中心区域数据，获得上述图像的实例分割结果，可以实现各个实例分割模型的优势互补，不再要求各个模型具有相同结构或含义的数据输出，在实例分割问题中取得更高的精度。
本公开实施例还提供一种计算机存储介质，其中，该计算机存储介质用于存储计算机程序，该计算机程序使得计算机执行如上述方法实施例中记载的任何一种图像处理方法的部分或全部步骤。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本公开并不受所描述的动作顺序的限制，因为依据本公开，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本公开所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的 部分，可以参见其他实施例的相关描述。
在本公开所提供的几个实施例中，应该理解到，所揭露的装置，可通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元(模块)可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储器中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储器中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储器包括：U盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储器中，存储器可以包括：闪存盘、只读存储器、随机存取器、磁盘或光盘等。
以上对本公开实施例进行了详细介绍，本文中应用了具体个例对本公开的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本公开的方法及其核心思想；同时，对于本领域的一般技术人员，依据本公开的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本公开的限制。

Claims (20)

  1. 一种图像处理方法，包括：
    对第一图像进行处理，获得所述第一图像中多个像素点各自的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实例中心之间的相对位置；
    基于所述多个像素点中每个像素点的所述语义预测结果和所述中心相对位置预测结果，确定所述第一图像的实例分割结果。
  2. 根据权利要求1所述的图像处理方法，其特征在于，在对所述第一图像进行处理之前，还包括：
    对第二图像进行预处理，得到所述第一图像，以使得所述第一图像满足预设对比度和/或预设灰度值。
  3. 根据权利要求1或2所述的图像处理方法，其特征在于，基于所述多个像素点中每个像素点的所述语义预测结果和所述中心相对位置预测结果，确定所述第一图像的所述实例分割结果，包括：
    基于所述多个像素点中每个像素点的所述语义预测结果，从所述多个像素点中确定位于所述实例区域的至少一个第一像素点；
    针对每个所述第一像素点，基于所述第一像素点的所述中心相对位置预测结果，确定所述第一像素点所属的实例。
  4. 根据权利要求3所述的图像处理方法，其特征在于，所述预测结果还包括中心区域预测结果，所述中心区域预测结果指示所述像素点是否位于实例中心区域，
    所述方法还包括：基于所述多个像素点中每个像素点的所述中心区域预测结果，确定所述第一图像的至少一个实例中心区域；
    基于所述第一像素点的中心相对位置预测结果，确定所述第一像素点所属的实例，包括：基于所述第一像素点的所述中心相对位置预测结果，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
  5. 根据权利要求4所述的图像处理方法，其特征在于，基于所述多个像素点中每个像素点的所述中心区域预测结果，确定所述第一图像的至少一个实例中心区域，包括：
    基于所述多个像素点中每个像素点的中心区域预测结果，对所述第一图像进行连通域搜索处理，得到至少一个实例中心区域。
  6. 根据权利要求4或5所述的图像处理方法，其特征在于，基于所述第一像素点的所述中心相对位置预测结果，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域，包括：
    基于所述第一像素点的位置信息和所述第一像素点的所述中心相对位置预测结果，确定所述第一像素点的中心预测位置，所述中心预测位置表示预测的所述第一像素点所属的实例中心区域的中心位置；
    基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域。
  7. 根据权利要求6所述的图像处理方法，其特征在于，基于所述第一像素点的中心预测位置和所述至少一个实例中心区域的位置信息，从所述至少一个实例中心区域中确定所述第一像素点对应的实例中心区域，包括：
    响应于所述第一像素点的中心预测位置属于所述至少一个实例中心区域中的第一实例中心区域，将所述第一实例中心区域确定为所述第一像素点对应的实例中心区域；或者
    响应于所述第一像素点的中心预测位置不属于所述至少一个实例中心区域中的任 意实例中心区域，将所述至少一个实例中心区域中与所述第一像素点的中心预测位置距离最近的实例中心区域确定为所述第一像素点对应的实例中心区域。
  8. 根据权利要求4-7任一项所述的图像处理方法，其特征在于，所述对第一图像进行处理，获得所述第一图像中多个像素点的预测结果，包括：
    对所述第一图像进行处理，得到所述第一图像中多个像素点各自的中心区域预测概率；
    基于第一阈值对所述多个像素点各自的中心区域预测概率进行二值化处理，得到所述多个像素点中每个像素点的中心区域预测结果。
  9. 一种电子设备，包括：
    预测模块，用于对第一图像进行处理，获得所述第一图像中多个像素点各自的预测结果，所述预测结果包括语义预测结果和中心相对位置预测结果，其中，所述语义预测结果指示所述像素点位于实例区域或背景区域，所述中心相对位置预测结果指示所述像素点与实例中心之间的相对位置；和
    分割模块，用于基于所述多个像素点中每个像素点的所示语义预测结果和所示中心相对位置预测结果，确定所述第一图像的实例分割结果。
  10. 根据权利要求9所述的电子设备，其特征在于，所述分割模块包括：
    第一单元，用于基于所述多个像素点中每个像素点的语义预测结果，从所述多个像素点中确定位于实例区域的至少一个第一像素点；
    第二单元，用于基于每个所述第一像素点的所述中心相对位置预测结果，确定所述每个第一像素点所属的实例。
  11. 一种图像处理方法，包括：
    获取N组实例分割输出数据，其中，所述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且所述N组实例分割输出数据具有不同的数据结构，所述N为大于1的整数；
    基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，其中，所述集成语义数据指示所述图像中位于实例区域的像素点，所述集成中心区域数据指示所述图像中位于实例中心区域的像素点；
    基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果。
  12. 根据权利要求11所述的图像处理方法，其特征在于，基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，包括：
    针对所述N个实例分割模型中的每个实例分割模型，基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据；
    基于所述N个实例分割模型中每个实例分割模型的语义数据和中心区域数据，得到所述图像的集成语义数据和集成中心区域数据。
  13. 根据权利要求12所述的图像处理方法，其特征在于，基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据，包括：
    基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像的多个像素点中每个像素点对应的实例标识信息；
    基于所述实例分割模型中所述多个像素点中每个像素点对应的实例标识信息，得到所述每个像素点在所述实例分割模型中的语义预测值，其中，所述实例分割模型的语义数据包括所述图像的多个像素点中每个像素点的语义预测值。
  14. 根据权利要求12或13所述的图像处理方法，其特征在于，基于所述实例分割模型的实例分割输出数据，得到所述实例分割模型的语义数据和中心区域数据，还包括：
    基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点；
    基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定所述实例分割模型的实例中心位置；
    基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域。
  15. 根据权利要求14所述的图像处理方法，其特征在于，
    在基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点之前，所述方法还包括：对所述实例分割模型的实例分割输出数据进行腐蚀处理，得到所述实例分割模型的腐蚀数据；
    基于所述实例分割模型的实例分割输出数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点，包括：
    基于所述实例分割模型的腐蚀数据，确定在所述实例分割模型中，所述图像中位于实例区域的至少两个像素点。
  16. 根据权利要求14或15所述的图像处理方法，其特征在于，基于所述实例分割模型中位于实例区域的至少两个像素点的位置信息，确定所述实例分割模型的实例中心位置，包括：
    将所述位于实例区域的至少两个像素点的位置的平均值，作为所述实例分割模型的实例中心位置。
  17. 根据权利要求14至16中任一项所述的图像处理方法，其特征在于，基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述实例分割模型的实例中心区域，包括：
    基于所述实例分割模型的实例中心位置和所述至少两个像素点的位置信息，确定所述至少两个像素点与所述实例中心位置的最大距离；
    基于所述最大距离，确定第一阈值；
    将所述至少两个像素点中与所述实例中心位置之间的距离小于或等于所述第一阈值的像素点确定为实例中心区域的像素点。
  18. 一种电子设备，包括：
    获取模块，用于获取N组实例分割输出数据，其中，所述N组实例分割输出数据分别为N个实例分割模型对图像进行处理获得的实例分割输出结果，且所述N组实例分割输出数据具有不同的数据结构，所述N为大于1的整数；
    转换模块，用于基于所述N组实例分割输出数据，得到所述图像的集成语义数据和集成中心区域数据，其中，所述集成语义数据指示所述图像中位于实例区域的像素点，所述集成中心区域数据指示所述图像中位于实例中心区域的像素点；
    分割模块，用于基于所述图像的集成语义数据和集成中心区域数据，获得所述图像的实例分割结果。
  19. 一种电子设备，包括处理器以及存储器，所述存储器用于存储计算机程序，所述计算机程序被配置成由所述处理器执行，所述处理器用于执行如权利要求1-8、11-18中任一项所述的方法。
  20. 一种计算机可读存储介质，所述计算机可读存储介质用于存储计算机程序，其中，所述计算机程序使得计算机执行如权利要求1-8、11-18中任一项所述的方法。
PCT/CN2019/105787 2018-09-15 2019-09-12 图像处理方法、电子设备及存储介质 Ceased WO2020052668A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
SG11202013059VA SG11202013059VA (en) 2018-09-15 2019-09-12 Image processing method, electronic device, and storage medium
JP2020564310A JP7026826B2 (ja) 2018-09-15 2019-09-12 画像処理方法、電子機器および記憶媒体
US17/135,489 US20210118144A1 (en) 2018-09-15 2020-12-28 Image processing method, electronic device, and storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201811077349.X 2018-09-15
CN201811077349.XA CN109389129B (zh) 2018-09-15 2018-09-15 一种图像处理方法、电子设备及存储介质
CN201811077358.9A CN109345540B (zh) 2018-09-15 2018-09-15 一种图像处理方法、电子设备及存储介质
CN201811077358.9 2018-09-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/135,489 Continuation US20210118144A1 (en) 2018-09-15 2020-12-28 Image processing method, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2020052668A1 true WO2020052668A1 (zh) 2020-03-19

Family

ID=69778202

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/105787 Ceased WO2020052668A1 (zh) 2018-09-15 2019-09-12 图像处理方法、电子设备及存储介质

Country Status (4)

Country Link
US (1) US20210118144A1 (zh)
JP (1) JP7026826B2 (zh)
SG (1) SG11202013059VA (zh)
WO (1) WO2020052668A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112085067A (zh) * 2020-08-17 2020-12-15 浙江大学 一种高通量筛选dna损伤反应抑制剂的方法
CN112966691A (zh) * 2021-04-14 2021-06-15 重庆邮电大学 基于语义分割的多尺度文本检测方法、装置及电子设备
CN113392793A (zh) * 2021-06-28 2021-09-14 北京百度网讯科技有限公司 用于识别车道线的方法、装置、设备、存储介质以及无人车
CN114550129A (zh) * 2022-01-26 2022-05-27 江苏联合职业技术学院苏州工业园区分院 一种基于数据集的机器学习模型处理方法和系统
CN114742999A (zh) * 2022-03-18 2022-07-12 北京航空航天大学 一种深度三网络半监督语义分割方法和系统
CN116324901A (zh) * 2020-09-09 2023-06-23 皇家飞利浦有限公司 确定目标对象类型和位置

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020206362A1 (en) * 2019-04-04 2020-10-08 Inscopix, Inc. Multi-modal microscopic imaging
CN110335277B (zh) * 2019-05-07 2024-09-10 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机可读存储介质和计算机设备
CN111429380B (zh) * 2020-04-08 2023-11-03 京东科技信息技术有限公司 一种图像矫正方法、装置、电子设备及存储介质
CN112329660B (zh) * 2020-11-10 2024-05-24 浙江商汤科技开发有限公司 一种场景识别方法、装置、智能设备及存储介质
CN113358739B (zh) * 2021-06-30 2024-07-09 常州工学院 一种基于双目图像融合技术的磁粉探伤检测装置
CN114936996B (zh) * 2021-09-23 2025-05-30 腾讯音乐娱乐科技（深圳）有限公司 一种图像检测方法、装置、智能设备及存储介质
CN113870247B (zh) * 2021-10-15 2024-09-10 杭州柳叶刀机器人有限公司 股骨头中心确定方法、装置、终端设备和可读存储介质
CN113963322B (zh) * 2021-10-29 2023-08-25 北京百度网讯科技有限公司 一种检测模型训练方法、装置及电子设备
CN114359907B (zh) * 2021-12-10 2025-12-02 阿里巴巴达摩院(杭州)科技有限公司 语义分割方法、车辆控制方法、电子设备及存储介质
CN114429475B (zh) * 2021-12-24 2024-12-24 北京达佳互联信息技术有限公司 图像处理方法、装置、设备及存储介质
CN114463546B (zh) * 2022-02-11 2024-12-20 淘宝（中国）软件有限公司 图像分割方法及装置
CN114550117B (zh) * 2022-02-21 2025-11-21 京东鲲鹏(江苏)科技有限公司 一种图像检测方法和装置
CN116758259B (zh) * 2023-04-26 2024-09-03 中国公路工程咨询集团有限公司 一种公路资产信息识别方法及系统
CN117830876B (zh) * 2023-11-21 2025-06-27 三峡大学 一种基于矢量回归和注意力机制的遥感旋转飞机检测算法
CN117853932B (zh) * 2024-03-05 2024-05-14 华中科技大学 一种基于光电吊舱的海面目标检测方法、检测平台及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102324092A (zh) * 2011-09-09 2012-01-18 华南理工大学 一种数字图像中颗粒状对象的自动分割方法
EP2871613A1 (en) * 2012-07-05 2015-05-13 Olympus Corporation Cell division process tracing device and method, and storage medium which stores computer-processable cell division process tracing program
CN107967688A (zh) * 2017-12-21 2018-04-27 联想(北京)有限公司 对图像中的对象进行分割的方法及系统
CN109345540A (zh) * 2018-09-15 2019-02-15 北京市商汤科技开发有限公司 一种图像处理方法、电子设备及存储介质
CN109389129A (zh) * 2018-09-15 2019-02-26 北京市商汤科技开发有限公司 一种图像处理方法、电子设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102324092A (zh) * 2011-09-09 2012-01-18 华南理工大学 一种数字图像中颗粒状对象的自动分割方法
EP2871613A1 (en) * 2012-07-05 2015-05-13 Olympus Corporation Cell division process tracing device and method, and storage medium which stores computer-processable cell division process tracing program
CN107967688A (zh) * 2017-12-21 2018-04-27 联想(北京)有限公司 对图像中的对象进行分割的方法及系统
CN109345540A (zh) * 2018-09-15 2019-02-15 北京市商汤科技开发有限公司 一种图像处理方法、电子设备及存储介质
CN109389129A (zh) * 2018-09-15 2019-02-26 北京市商汤科技开发有限公司 一种图像处理方法、电子设备及存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112085067A (zh) * 2020-08-17 2020-12-15 浙江大学 一种高通量筛选dna损伤反应抑制剂的方法
CN116324901A (zh) * 2020-09-09 2023-06-23 皇家飞利浦有限公司 确定目标对象类型和位置
CN112966691A (zh) * 2021-04-14 2021-06-15 重庆邮电大学 基于语义分割的多尺度文本检测方法、装置及电子设备
CN112966691B (zh) * 2021-04-14 2022-09-16 重庆邮电大学 基于语义分割的多尺度文本检测方法、装置及电子设备
CN113392793A (zh) * 2021-06-28 2021-09-14 北京百度网讯科技有限公司 用于识别车道线的方法、装置、设备、存储介质以及无人车
CN114550129A (zh) * 2022-01-26 2022-05-27 江苏联合职业技术学院苏州工业园区分院 一种基于数据集的机器学习模型处理方法和系统
CN114742999A (zh) * 2022-03-18 2022-07-12 北京航空航天大学 一种深度三网络半监督语义分割方法和系统

Also Published As

Publication number Publication date
SG11202013059VA (en) 2021-02-25
US20210118144A1 (en) 2021-04-22
JP2021512446A (ja) 2021-05-13
JP7026826B2 (ja) 2022-02-28

Similar Documents

Publication Publication Date Title
US20210118144A1 (en) Image processing method, electronic device, and storage medium
TWI777092B (zh) 一種圖像處理方法、電子設備及存儲介質
CN110428432B (zh) 结肠腺体图像自动分割的深度神经网络算法
Zhang et al. Automated semantic segmentation of red blood cells for sickle cell disease
CN110490850B (zh) 一种肿块区域检测方法、装置和医学图像处理设备
TWI786330B (zh) 一種圖像處理方法、電子設備及存儲介質
CN110853022B (zh) 病理切片图像的处理方法、装置、系统及存储介质
CN108765278B (zh) 一种图像处理方法、移动终端及计算机可读存储介质
Bozorgtabar et al. Skin lesion segmentation using deep convolution networks guided by local unsupervised learning
CN109410168B (zh) 用于确定图像中的子图块类别的卷积神经网络的建模方法
CN114240961B (zh) 一种U-Net++细胞分割网络系统、方法、设备及终端
CN110570352A (zh) 图像标注方法、装置、系统及细胞标注方法
CN110046574A (zh) 基于深度学习的安全帽佩戴识别方法及设备
CN113781387A (zh) 模型训练方法、图像处理方法、装置、设备及存储介质
Zhang et al. Artifact detection in endoscopic video with deep convolutional neural networks
CN120318171A (zh) 一种利用宫颈细胞病理图像直接进行hpv预测的方法、装置、电子设备、存储介质及程序
CN111339899B (zh) 导管特征获取方法、装置、设备、介质和智能显微镜
Abosinnee et al. Edges in image with illumination variations scenarios: a review: AS Abosinnee et al.
CN114387489A (zh) 电力设备识别方法、装置和终端设备
CN111862003B (zh) 医疗影像目标信息获取方法、装置、设备和存储介质
CN116189209B (zh) 医疗文档图像分类方法和装置、电子设备和存储介质
US12079950B2 (en) Image processing method and apparatus, smart microscope, readable storage medium and device
CN113706449A (zh) 基于病理图像的细胞分析方法、装置、设备及存储介质
Saranya et al. Revolutionizing Technology: Advancements, Challenges, and Future of Computer Vision
CN121120653B (zh) 病理图像干燥综合征自动诊断方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19860875

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020564310

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19860875

Country of ref document: EP

Kind code of ref document: A1