WO2020078263A1 - 图像分割方法、装置、计算机设备及存储介质 - Google Patents

图像分割方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2020078263A1
WO2020078263A1 PCT/CN2019/110541 CN2019110541W WO2020078263A1 WO 2020078263 A1 WO2020078263 A1 WO 2020078263A1 CN 2019110541 W CN2019110541 W CN 2019110541W WO 2020078263 A1 WO2020078263 A1 WO 2020078263A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
segmentation
module
image segmentation
segmentation module
Prior art date
Application number
PCT/CN2019/110541
Other languages
English (en)
French (fr)
Inventor
陈思宏
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP19874073.0A priority Critical patent/EP3869456A4/en
Priority to JP2021513795A priority patent/JP7085062B2/ja
Priority to KR1020217005619A priority patent/KR102597385B1/ko
Publication of WO2020078263A1 publication Critical patent/WO2020078263A1/zh
Priority to US17/173,172 priority patent/US12002212B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10104—Positron emission tomography [PET]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Definitions

  • the present application relates to the field of computer technology, and in particular, to an image segmentation method, device, computer equipment, and storage medium.
  • image segmentation technology is more and more widely used, for example, medical image segmentation, natural image segmentation and so on.
  • the image segmentation technology refers to the technology that divides the image into several specific areas with unique properties and proposes the target of interest. For example, in a human tissue image segmentation scene, medical images can be segmented, so that the segmented images can clearly distinguish various tissues of the human body.
  • Cascaded 3D FCN Cascaded Three-dimensional Fully Convolutional Networks
  • Pspnet Pyramid Scene Parsing Network
  • Cascaded3D FCN is a three-dimensional (3D) network, which is mainly used to segment abdominal tissues.
  • Pspnet is a two-dimensional (2D) network, which is mainly used to segment natural images.
  • the above image segmentation methods usually require technicians to analyze certain human tissue images for image segmentation needs, analyze which human tissue is to be segmented in the image, and what characteristics of the pixel distribution of such human tissue images, and according to Analyze the results to design a model, so as to obtain a sample image of this human tissue to train the designed model, so that the trained model can be used to segment the image that needs to be segmented.
  • an image segmentation method includes:
  • the computer device pre-trains the first initial model based on multiple first sample images to obtain a second initial model.
  • the multiple first sample images include multiple images of human tissue
  • the second initial model includes Distribution information of multiple target areas corresponding to the multiple human tissues;
  • the computer device trains the second initial model based on a plurality of second sample images to obtain an image segmentation model, the plurality of second sample images are target human tissue images, and the image segmentation model is acquired during training Obtaining image information of the plurality of second sample images, the image information of the plurality of second sample images at least includes distribution information of a plurality of target regions corresponding to the target human tissue;
  • the computer device calls the image segmentation model, and the image segmentation model segments the first image according to the image information and outputs a second image.
  • an image segmentation device including:
  • the training module is used to pre-train the first initial model based on multiple first sample images to obtain a second initial model.
  • the multiple first sample images include multiple human tissue images, and the second initial
  • the model includes distribution information of multiple target regions corresponding to the multiple human tissues;
  • the training module is also used to train the second initial model based on multiple second sample images to obtain an image segmentation model.
  • the multiple second sample images are target human tissue images.
  • the image segmentation model acquires image information of the plurality of second sample images, and the image information of the plurality of second sample images at least includes distribution information of a plurality of target regions corresponding to the target human tissue;
  • the segmentation module is configured to call the image segmentation model when the first image to be segmented is acquired, and the image segmentation model segments the first image according to the image information and outputs a second image.
  • a computer device in one aspect, includes a processor and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor Perform the method described in the above embodiment.
  • one or more non-volatile storage media storing computer-readable instructions are provided.
  • the computer-readable instructions are executed by one or more processors, the one or more processors execute the foregoing embodiments The method.
  • FIG. 1 is an implementation environment of an image segmentation method provided by an embodiment of the present application
  • FIG. 2 is a flowchart of an image segmentation model training method provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a processing flow of a modal fusion module provided by an embodiment of the present application
  • FIG. 4 is a schematic diagram of an image sampling method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an image sampling method provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a 3D model provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an image post-processing method provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an image post-processing method provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an image segmentation model provided by an embodiment of the present application.
  • FIG. 10 is a flowchart of an image segmentation method provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an image segmentation device according to an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • 15 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 1 is an implementation environment of an image segmentation method provided by an embodiment of the present application.
  • the implementation environment may include multiple computer devices.
  • the multiple computer devices may implement data interaction through a wired connection, or may implement data interaction through a wireless network connection, which is not limited in the embodiments of the present application.
  • the computer device 101 may be used to segment the image.
  • the image may be a medical image, that is, a human tissue image
  • the image segmentation method provided in the embodiment of the present application may be applied
  • human tissue image segmentation scenarios for example, liver cancer segmentation, brain cancer and peripheral injury segmentation, lung cancer segmentation, pancreatic cancer segmentation, colorectal cancer segmentation, liver invasion microvascular segmentation, hippocampal structure segmentation, prostate structure segmentation, left atrium segmentation, pancreas segmentation
  • the segmentation scenes of human tissue images such as the segmentation of the liver, the segmentation of the liver, or the segmentation of the spleen, of course, may also be other human tissue image segmentation scenes.
  • the image can also be other types of images, and the image segmentation method can also be applied to other image segmentation scenes, for example, landscape image segmentation scenes.
  • the computer device 102 can be used to collect images, and send the collected images to the computer device 101, and the computer device 101 provides an image segmentation service.
  • the computer device 101 may also collect images and segment the collected images, which is not limited in the embodiments of the present application.
  • the computer device 102 may also be used to store images obtained from other computer devices, and the computer device 101 may obtain the stored images from the computer device 102 for segmentation.
  • both the computer device 101 and the computer device 102 may be provided as a terminal or a server, which is not limited in the embodiments of the present application.
  • the image segmentation model training method can be applied to a computer device, which can be the computer device 101 in the above-described implementation environment, or Other computer equipment. That is, the image segmentation model can be trained on the computer device 101, or the image segmentation model can be trained on other computer devices, the image segmentation model is processed into a configuration file, and the configuration file is sent to the computer device 101. Then, the computer device 101 stores an image segmentation model.
  • the computer device 101 may also call an image segmentation model trained on other computer devices when there is an image segmentation requirement, which is not limited in the embodiments of the present application. Referring to FIG. 2, the method may include the following steps:
  • the computer device pre-trains the first initial model based on the plurality of first sample images to obtain the second initial model.
  • the plurality of first sample images include various human tissue images, for example, the plurality of first sample images may include liver images, brain images, lung images, pancreas images, large intestine images, hippocampus Images of parts of the body, images of parts of the prostate, images of parts of the heart, images of the spleen, and other human tissue images, of course, images of other parts can also be included. Only an exemplary description is provided here. The number and specific types of tissue images are limited.
  • the computer device obtains the model parameters of the second initial model based on the pre-training of multiple human tissue images, and the second initial model includes distribution information of multiple target regions corresponding to the multiple human tissues, and It is through the segmentation of a variety of human tissue images to obtain a priori knowledge.
  • pre-training the model through the data of different scenes can make the second initial model have the relevant knowledge of segmenting the human tissue image, and can be used to simulate the rotation of the medical student in various departments, so that the medical student can have a certain medical Knowledge or clinical knowledge.
  • the second initial model already has certain prior knowledge, and when a certain human tissue image needs to be segmented later, the sample image of this human tissue can be directly used to train the second initial model without the need for correlation
  • the technician analyzes this human tissue image and redesigns the model, that is, the second initial model has certain medical knowledge, and various human tissue images can be directly trained using the above second initial model, which can effectively improve The practicability and versatility of the second initial model and the image segmentation model trained based on the second initial model.
  • the computer device acquires a plurality of second sample images, and each second sample image carries a label, and the label is used to indicate a target segmentation result of the second sample image.
  • the computer device can train to obtain an image segmentation model based on multiple second sample images.
  • the computer device can store the multiple second sample images, which can be obtained when image segmentation model training is required. get.
  • each second sample image may also carry a label indicating the target segmentation result, where the target segmentation result refers to the correct segmentation result of the second sample image, or refers to the second sample image Real segmentation results.
  • the plurality of second sample images may also be stored in other computer equipment, and the computer equipment may be obtained from other computer equipment when the image segmentation model training is required, which is not limited in the embodiments of the present application.
  • the plurality of second sample images may be stored in an image database, and each second sample image also carries a label. Then, this step 201 can obtain multiple second sample images from the image database for the computer device.
  • the computer device inputs the plurality of second sample images into the second initial model.
  • the computer device may directly train the second initial model, that is, perform step 201, and then perform step 202, accordingly, in this step 202, the computer device acquires multiple second After the sample image, the second initial model can be trained based on the plurality of second sample images to obtain an image segmentation model, so that the obtained first image can be accurately segmented based on the image segmentation model later.
  • the computer device may also store the second initial model, or may send the second initial model to other computer devices, and other computer devices may perform the subsequent model training process based on the second initial model Then, correspondingly, in step 202, after acquiring multiple second sample images, the computer device may call the second initial model and input the multiple second sample images into the second initial model.
  • This embodiment of the present application There is no limitation on the specific implementation method.
  • the second initial model may be stored in the computer device in step 202, and may be directly called when the image segmentation model training is needed, or directly after step 200, based on the obtained second initial model, Without calling.
  • the second initial model may also be stored in other computer equipment. When the computer equipment needs to perform image segmentation model training, it may be called from other computer equipment, which is not limited in the embodiments of the present application.
  • model parameters of the second initial model are initial values
  • the computer device may use the plurality of second sample images as training samples and verification samples to train the second initial model, that is, through the second sample images Adjust the model parameters of the second initial model, so that the model parameters after multiple adjustments can obtain a more accurate segmentation result when segmenting the first image.
  • the computer device inputs the plurality of second sample images into the second initial model, and each second sample image may be segmented by the second initial model, based on the segmentation result of the second initial model and the The label, which is the target segmentation result of the second sample image, determines the segmentation ability of the second initial model, so that the model parameters of the second initial model can be adjusted to continuously improve the segmentation ability of the second initial model, so that subsequent training The obtained image segmentation model can be accurately segmented.
  • the second initial model in the computer device acquires image information of the plurality of second sample images.
  • the image information includes at least distribution information of a plurality of target regions corresponding to the target human tissue. That is, the second initial model can first obtain image information of multiple second sample images, so as to obtain certain prior knowledge, and understand the distribution information of multiple target regions corresponding to the target human tissue that needs to be segmented, for example,
  • the distribution information may include the distribution type of a plurality of target regions, the number of target regions, and the size range of the target regions. Of course, it may also include other information, which is not enumerated here in the embodiments of the present application.
  • the second initial model may be based on the relevant conditions of the plurality of second sample images, and initially learn some segmentation laws and processing laws of the first image that need to be segmented after subsequent model training is completed.
  • the second initial model may perform connected domain processing on the plurality of second sample images based on the information in the tags of the plurality of second sample images, and perform topological analysis on the plurality of second sample images after the connected domain processing To obtain the distribution information of multiple target areas corresponding to the target human tissue.
  • the target area refers to an area where pixels of a target type in the plurality of second sample images are located.
  • the image information may further include attribute information of each second sample image, so that based on the attribute information of the second sample image, the second sample image is preprocessed so that some of the second sample image
  • the basic information is more uniform or more accurate, so that the second sample image is more accurate when segmented.
  • the segmented image can also be post-processed to make the segmentation result more accurate.
  • the attribute information may include a gray scale range, a number of modes, and a size range, etc., which are not limited in the embodiments of the present application.
  • both the first initial model and the second initial model may include a first image segmentation module and a second image segmentation module, where the first image segmentation module and the second image segmentation module respectively correspond to a segmentation algorithm
  • the first image segmentation module is used to segment a three-dimensional (3D) image
  • the second image segmentation module is used to segment a two-dimensional (2D) image.
  • the image segmentation model trained based on the second initial model may also include two image segmentation modules.
  • the image segmentation model obtained in this way can directly segment the three-dimensional image or convert the three-dimensional image into a two-dimensional image.
  • the two-dimensional image can also be directly segmented, and through two segmentation algorithms, you can flexibly choose different
  • the image segmentation method can effectively improve the practicality of the image segmentation model, and can also improve the accuracy of image segmentation.
  • the image information acquired by the computer device may further include the number of images of the plurality of second sample images, so that the image segmentation sub-module as the second image segmentation module may be selected based on the number of images, specifically,
  • the image segmentation sub-module as the second image segmentation module may be selected based on the number of images, specifically,
  • the processing method for the second sample image based on the number of images.
  • the image information obtained in step 203 includes the distribution information of multiple target regions corresponding to the target human tissue, the attribute information of each second sample image, and the multiple second samples.
  • the number of images of the image, and the first initial model, the second initial model, and the image segmentation model include the first image segmentation module and the second image segmentation module as an example for description.
  • the second initial model in the computer device preprocesses each second sample image according to the image information, and inputs each preprocessed second sample image into the first image segmentation module in the second initial model And the second image segmentation module.
  • the step 204 is to preprocess each second sample image based on the attribute information of each second sample image, and input the preprocessed multiple second sample images to the first image segmentation module and the second image
  • the process in the segmentation module, that is, the data on which the preprocessing is performed in step 204 is attribute information of each second sample image in the image information.
  • the second initial model can preprocess the second sample image so that the preprocessed second sample image meets the image segmentation conditions of the first image segmentation module and the second image segmentation module, and can also remove the Abnormal pixels, or normalizing the pixels of the first image, etc., so that the preprocessed second sample image is more accurate when the image is segmented.
  • the attribute information of the second sample image is different, and the preprocessing process may also be different.
  • the preprocessing process may include any one or more of the following steps:
  • Step 1 When it is determined that there is an abnormal pixel in the second sample image according to the attribute information, the second initial model deletes the abnormal pixel.
  • the abnormal pixel point refers to a pixel point whose pixel value is an abnormal value, wherein the pixel value of the abnormal pixel point is the average of the multiple pixel values among the multiple pixel values of the second sample image Pixel values that deviate by more than two standard deviations.
  • a pixel value whose deviation from the average value exceeds three times the standard deviation may be a pixel value of an abnormal pixel having an abnormal height. If there are abnormal pixel points in the second sample image, the abnormal pixel points in the second sample image can be deleted to avoid the influence of the abnormal pixel points on the segmentation result, thereby ensuring the segmentation result of the second sample image more acurrate.
  • the gray curve of the second sample image can be obtained according to the pixel values of the multiple pixel points of the second sample image, so that the pixels of the second sample image can be obtained from the gray curve The average value, maximum value or minimum value of the pixel value, etc., to determine the abnormal pixel value.
  • the pixel value of the second sample image can also be counted in the form of a histogram to determine the abnormal pixel value, for example, The pixel point corresponding to the pixel value whose average value difference is greater than the preset difference value may be used as an abnormal pixel point, or 80% of the multiple pixel values of the second sample image may be used according to the difference value from the average value As a normal value, 20% is taken as an abnormal value, wherein the abnormal value is a pixel value with a large difference from the average value, and the normal value is a pixel value with a small difference from the average value.
  • step 1 can be implemented by any kind of outlier detection algorithm or any kind of outlier processing method, which is not made in the embodiments of the present application. limited.
  • Step 2 When it is determined according to the attribute information that the gray range of the second sample image after the abnormal pixel is deleted is greater than the target range, the second initial model normalizes the second sample image and grays the second sample image. The degree range is adjusted to be within the target range.
  • a target range may also be set in the second initial model, and the target range may be set in advance by related technical personnel, for example, the target range may be [0,255].
  • the gray range of the second sample image is greater than the target range, it can be normalized to adjust the gray range of the second sample image within the target range, so that all the second sample images in the subsequent segmentation process
  • the gray scale ranges are within the target range, the gray scale range is uniform, and the second sample images are also comparable, and the segmentation results obtained by segmenting the second sample image are more accurate.
  • the normalization process may be implemented by any normalization method.
  • the gray scale range may be converted into the target range according to a linear function, which is not limited in this embodiment of the present application and will not be described in detail.
  • Step 3 When it is determined that the number of channels of the second sample image is greater than one according to the attribute information, the second initial model subtracts the average value of the target image from each pixel value of the second sample image.
  • the second sample image may be a color image or a grayscale image, where the number of channels of the color image is greater than one, and the number of channels of the grayscale image is one. If it is determined that the second sample image is a color image according to the attribute information, each pixel value of the second sample image may be subtracted from the average value of the target image.
  • the target image mean can be obtained during the pre-training of the second initial model, that is, the target image mean can be obtained in step 200 above.
  • the average value of the target image may be the average value of multiple first sample images during pre-training, that is, the average value of the pixel values.
  • the range of pixel values (grayscale range) of the second sample image can be made to be consistent with the range of pixel values (grayscale range) of the first sample image during model pre-training, so that model training and model Performing the above processing on the image during use can make the gray scale range of the processed image consistent, so that the image segmentation result is more accurate.
  • Step 4 When it is determined that the number of modalities of the second sample image is greater than one according to the attribute information, the second initial model inputs the second sample image to the modal fusion module, and the modal fusion module selects A plurality of pixel values are filtered to obtain a target number of pixel values of the preprocessed second sample image, and the number of modalities of the preprocessed second sample image is one.
  • the number of modalities of the second sample image may not be one, for example, the second sample image is obtained through various imaging principles or various imaging devices, such as electronic computed tomography (Computed Tomography, CT), magnetic resonance imaging (MagneticResonanceImaging, MRI), Positron EmissionComputed Tomography (PET), etc.
  • the second initial model can also perform modal fusion on the multi-modal image, so as to segment the fused image.
  • the modal fusion module is a module in the second initial model. When the number of modalities of the second sample image is greater than one, the modal fusion module can perform modal fusion on the second sample image.
  • the processing procedure of the second sample image by the modal fusion module in step 4 can also be understood as: the modal fusion module can select the target number of pixel values from the multiple pixel values of the second sample image according to the module parameters As the pixel value of the second sample image.
  • the target number is the number of pixel values of the second sample image with one mode number.
  • the module parameters of the modal fusion module can be adjusted during the model training process, so that the pixel values of the selected target number are more representative and can better characterize the characteristics of the second sample image.
  • the above only provides one modal fusion method. Specifically, the process may also adopt other methods, which are not limited in the embodiments of the present application.
  • the number of modalities of the second sample image may be n
  • n is a positive integer
  • the modal fusion module may fuse the pixel values of the n modalities and perform
  • the above selection step finally obtains the pixel values of the second sample image to be input into the first image segmentation module and the second segmentation module, thereby fusing the n-modal second sample images into one modal second sample image.
  • modality is modal
  • H height Height
  • W width Width
  • D depth Depth
  • C is channel number Channel
  • Concat stacking function
  • Input is input
  • Convolution convolution
  • Featuremap is feature map.
  • the preprocessed second sample image may be input into the first image segmentation module and the second image segmentation module to perform subsequent segmentation steps.
  • the preprocessing process is not limited to the above four steps, and the preprocessing process may also include other steps, for example, the second initial model may sample the second sample image, and may be based on the size of the second sample image
  • the range determines the sampling method for the second sample image, for example, taking the downsampling factor of the second initial model set to 8 as an example, as shown in Figure 4, if the image is resized (Resize) after downsampling by 8 times the size If it is larger than one pixel, you can directly resize the image.
  • the first image segmentation module and the second image segmentation module respectively segment each second sample image to obtain a first segmentation result and a second segmentation result.
  • the first image segmentation module can adopt a 2-stage flow design, that is, the first image segmentation module can perform two-stage segmentation on the second sample image, that is, Two segmentation, the first image segmentation module can coarsely segment the second sample image in the first stage, and the second image segmentation module can finely segment the second sample image in the second stage, so that it can deal with different difficulty segmentation tasks .
  • the first image segmentation module may be implemented using an 18-layer 3D Unity Networking (unet) model.
  • FIG. 6 is a schematic structural diagram of a 3D model provided by an embodiment of the present application. As shown in FIG. 6, The 3D model may first perform feature extraction on the second sample image, and perform upsampling based on the extracted features.
  • the feature extraction process can be implemented through steps such as convolution and pooling. During the feature extraction process, the size of the second sample image gradually becomes smaller. During the upsampling process, the second sample image ’s previous feature extraction process can be synthesized.
  • Some data and the data obtained after feature extraction are up-sampled, that is, the above-mentioned up-sampling process is implemented by means of skip connection, and finally the segmentation process of the second sample image is realized.
  • [132,132,116] and [64,64,56] and so on represent the size of the image, including width, height and depth.
  • 32, 64, 128, 512, 258 + 512, etc. are the number of convolution kernels of the upper layer network, which will not be explained one by one here.
  • the embodiment of the present application only takes the 3D model as an example, specifically
  • the image segmentation module can also be implemented using other models, which is not limited in this embodiment of the present application and will not be described in detail.
  • the process of segmenting the second sample image by the first image segmentation module may be: the first image segmentation module classifies the second sample image twice based on the module parameters of the first image segmentation module to obtain the first segmentation result
  • the classification object of the first classification in the two classifications is all pixels of the second sample image, and the classification object of the second classification is the foreground pixels in the result of the first classification. In this way, the coarse segmentation is performed first, and then the fine segmentation is performed, so that the two segmentation results are synthesized to obtain the first segmentation result, which can effectively improve the accuracy of the image segmentation.
  • the first image segmentation module can support multi-type segmentation, and the first image segmentation module can classify each pixel of the second sample image to determine which type each pixel is, that is, at least two types A type.
  • the at least two types can be divided into two categories, one is the background and the other is the foreground.
  • the corresponding pixels are the background pixels and the foreground pixels respectively, that is, the pixels of the type background are the background Pixels, pixels of type foreground are pixels of foreground.
  • the background is one of at least two types
  • the foreground is one or more types other than the background in the at least two types.
  • the above two segmentation processes may include the following steps one to three:
  • Step 1 The first image segmentation module classifies each pixel of the second sample image based on the module parameters of the first image segmentation module to obtain a third segmentation result, and the third segmentation result is used to indicate the second Each pixel of the sample image has a probability of each of at least two types.
  • This step one is the rough segmentation process of the second sample image, which is the first stage in the 2-stage flow design.
  • the first image segmentation module can distinguish which pixels in the second sample image are foreground and which are background, so that the outer contour of the target area in the second sample image can be determined, so that the second step
  • the pixel points in the outer contour determined in step 1 are further classified, so that the types of each pixel point in the outer contour are more finely distinguished to determine the specific distribution of the target area.
  • Step 2 The first image segmentation module classifies each foreground pixel in the third segmentation result based on the third segmentation result and the module parameters of the first image segmentation module to obtain a fourth segmentation result.
  • the segmentation result is used to indicate the probability that each foreground pixel in the third segmentation result is each of the at least two types.
  • the second step is the fine segmentation of the second sample image, which is the second stage of the 2-stage flow design.
  • the first image segmentation module may classify the above-mentioned pixels determined as the foreground again, and determine again which of the at least two types each pixel is.
  • Step 3 The first image segmentation module obtains a first segmentation result based on the third segmentation result and the fourth segmentation result.
  • the first image segmentation module may also synthesize the segmentation results twice to determine the first segmentation result of the second sample image.
  • the segmentation result of the background pixel point in the third segmentation result may be used as the segmentation result of the pixel point in the first segmentation result.
  • the first image segmentation module can place this part of pixels in the third segmentation result The average of the division results in the fourth division result is taken as the first division result.
  • the first image segmentation module may also directly use the average value of the third segmentation result and the fourth segmentation result as the first segmentation result, where the fourth segmentation result further includes the The result of segmentation of background pixels. This part of pixels is directly determined as the background. That is, in the second stage, the first image segmentation module does not classify the background pixels in the first stage again, but directly assumes that the type of the background pixels is background.
  • the first image segmentation module may also perform weighted summation of the third segmentation result and the fourth segmentation result to obtain the first
  • the embodiment of the present application does not limit the specific implementation manner of step three.
  • the second image segmentation module may be implemented using a deep residual network (Deep residual network, ResNet).
  • ResNet Deep residual network
  • the ResNet may be ResNet-18, ResNet-101, or ResNet- 152.
  • the embodiment of the present application does not limit which ResNet is specifically adopted.
  • the second image segmentation module may include at least one image segmentation submodule, and different image segmentation submodules have different depths.
  • the second image segmentation module may include two image segmentation sub-modules: ResNet-18 and ResNet-101, where the depth of ResNet-101 is greater than the depth of ResNet-18.
  • the image information may also include the number of images of the plurality of second sample images. In this way, when the number of images of the second sample image is different, image segmentation sub-modules of different depths can be used for training, so that the problem of over-fitting or poor segmentation ability of the trained model can be avoided.
  • the second initial model may also store the correspondence between the number of images and the image segmentation submodule.
  • this step 205 further includes: the second initial model is based on the number of images of the plurality of second sample images, An image segmentation sub-module corresponding to the number of images is acquired as the second image segmentation module, and the number of images of the plurality of second sample images is one type of information in the image information acquired in step 203 above.
  • the greater the number of images the greater the depth of the acquired image segmentation sub-module. This can effectively deal with the situation of small data, and the model can be trained when the number of samples is small, to obtain an image segmentation model with good segmentation effect.
  • the acquiring step of the second image segmentation module may be: when the number of images of the plurality of second sample images is greater than a preset number, the second The initial model acquires the first image segmentation sub-module; when the number of images of the plurality of second sample images is less than or equal to the preset number, the second initial model acquires the second image segmentation sub-module.
  • the depth of the first image segmentation sub-module is greater than the depth of the second image segmentation sub-module.
  • the preset number may be preset by relevant technical personnel, and the specific value of the target number is not limited in the embodiment of the present application.
  • the first image segmentation sub-module may be ResNet-101
  • the second image segmentation sub-module may be ResNet-18.
  • the acquiring step of the second image segmentation module may be: when the second When the number of images of the sample image is less than 100, ResNet-18 can be used as the basic model, and when the number of images of the second sample image is greater than 100, ResNet-101 can be used as the basic model.
  • the structure tables of ResNet-18 and ResNet-101 are the following Table 1 and Table 2 respectively:
  • Conv1 can be a convolutional layer
  • the size of the convolution kernel is 7x7
  • the number of convolution kernels is 64
  • the step size is 2.
  • the first layer of Conv2_x is the pooling layer. After the pooling layer, there are two convolutional layers, both of which are 64 3x3 convolution kernels.
  • the two convolution kernels are a block.
  • the Conv2_x pooling layer includes Two blocks, that is, the Conv2_x includes one pooling layer and four convolutional layers. It should be noted that the above Table 1 and Table 2 only show the structure of Conv1 to Conv5_x. In fact, after Conv5_x, there is actually a Full Convolution FC layer that is not shown in Table 1 and Table 2. Not much to repeat here.
  • the first layer of Conv3_x that is, the step size of Conv3_1 is set to 2
  • the step size of the first layer of Conv4_x is set to 1
  • the dilation is set to 2, which can be avoided
  • the effect of downsampling on the segmentation results can also retain the receptive field of each layer of ResNet-101.
  • there is a linear rectification function (Reectified Linear Unit, ReLU) layer and a batch normalization (Batch Normalization) layer which will not be repeated here in the embodiments of the present application.
  • ResNet-18 and ResNet-101 Conv1 to Conv5_x are basic models, that is, the backbone model of the second image segmentation module, after Conv5_3, the second sample image can also be down-sampled, specific Ground, the down-sampling process can also use multi-scale convolution kernels, for example, you can use 1, 9, 19, 37 and 74 multiples of convolution kernels.
  • the down-sampling process is usually implemented by a pool layer. In the embodiment of the present application, all pool layers may be replaced with a depthwise convolution layer.
  • the above settings can also be set or adjusted by related technical personnel according to image segmentation requirements, which is not specifically limited in the embodiments of the present application.
  • the model parameters of the second initial model can be obtained by pre-training on multiple first sample images, that is, the parameters of Conv1 to Conv5_x can be obtained from pre-training of multiple first sample images,
  • the parameters of Conv1 to Conv5_x are mainly trained.
  • a Gaussian distribution value with a variance of 0.01 and a mean of 0 can be used as the initial value.
  • the initial value of the other layer may also be other numerical values, and the embodiment of the present application does not specifically limit the setting of the initial value.
  • the segmentation process of the second sample image by the second image segmentation module may include the following steps one and two:
  • Step 1 The second image segmentation module performs feature extraction on the second sample image based on the module parameters of the second image segmentation module.
  • the second image segmentation module may perform feature extraction on the second sample image based on the obtained module parameters of the second image segmentation module to obtain features of the second sample image, for example, the feature may be a feature map form.
  • Step 2 The second image segmentation module classifies each pixel of the second sample image based on the extracted features to obtain a second segmentation result.
  • the second image segmentation module After the second image segmentation module extracts features, it can also perform the above-mentioned downsampling process, and after all the information is combined, classify each pixel of the second sample image to determine the second segmentation result.
  • the second image segmentation module is used to segment a 2D image. If the second sample image is a 3D image, before the second image segmentation module segments the second sample image, the second initial The model also needs to process the second sample image and process the 3D image into a 2D image, so that the 2D image is input into the second image segmentation module.
  • the second initial model performs After processing, a plurality of first sub-images are obtained, and the first sub-images are two-dimensional images.
  • the process of processing a 3D image into multiple 2D images can use any 3D / 2D conversion method.
  • a 3D image can be sampled in a certain direction to obtain multiple 2D images.
  • the segmentation process of the first image by the second image segmentation module includes: based on the module parameters of the second image segmentation module, the second image segmentation module separately segments the first sub-images corresponding to the first image To obtain multiple second sub-segmentation results; the second image segmentation module fuse the multiple sub-segmentation results to obtain a second segmentation result.
  • the above steps 202 to 205 are to input the plurality of second sample images into the second initial model, obtain image information of the plurality of second sample images from the second initial model, and according to the image information, the second initial
  • the first image segmentation module and the second image segmentation module in the model, the process of segmenting each second sample image, the second initial model includes both a module for segmenting 3D images and a module for segmenting 2D
  • the image segmentation module improves the applicability and versatility of the second initial model.
  • the image segmentation model trained based on the second initial model is also more applicable and versatile, and provides a variety of flexible
  • the variable segmentation method improves the accuracy of image segmentation.
  • the second initial model in the computer device obtains the first segmentation error and the second segmentation error based on the labels of the plurality of second sample images, the first segmentation result, and the second segmentation result, respectively.
  • the second initial model may determine whether the first segmentation result and the second segmentation result are accurate based on the label of the second sample image, specifically, whether the segmentation result is accurate can be passed Segmentation error is measured.
  • the first segmentation error is the segmentation error of the first segmentation result corresponding to the first image segmentation module
  • the second segmentation error is the segmentation error of the second segmentation result corresponding to the second image segmentation module.
  • the segmentation error acquisition process of the first segmentation result is implemented using a first loss function
  • the first segmentation error acquisition process is implemented using a first loss function
  • each of the pixels in the first loss function The weight of the type is determined based on the proportion of pixels of the type in the image information of the plurality of second sample images in the plurality of second sample images. For example, the weight can be determined by the following formula:
  • N is the number of images of the second sample image
  • i is the identification of the second sample image
  • t c is the number of pixels of type c in the second sample image i
  • n i is The number of all pixels in the second sample image i
  • ⁇ is an accumulation function or a summation function.
  • the segmentation error acquisition process of the second segmentation result is implemented by a second loss function, and the weight of the second loss function is determined based on an online hard sample mining (Online Hard Example Mining, OHEM) algorithm, which can be effective Distinguish the difficult samples in the second sample image, and reduce the influence of these samples on the parameters of the model, so as to deal with the adverse effects caused by the imbalance of the sample labels.
  • OHEM Online Hard Example Mining
  • the second loss function may be a cross entropy function (cross entropy function), and the above first loss function may also be a cross entropy function or other loss functions.
  • the first loss function and the second loss function may be the same or different. Which loss function is specifically adopted for the first loss function and the second loss function, and the first loss function and the second Whether the loss functions are the same is not limited.
  • the second initial model in the computer device adjusts the module parameters of the first image segmentation module and the second image segmentation module based on the first segmentation error and the second segmentation error, respectively, until the first iteration stop count is reached Stop, and get the first image segmentation module and the second image segmentation module.
  • the second initial model can adjust the module parameters of the two image segmentation modules, so that the module parameters after multiple adjustments can make the first image segmentation module and The segmentation result of the second sample image by the second image segmentation module is more accurate.
  • the first iteration stop count is determined based on cross-validation.
  • the first iteration stop number may be determined based on the k-fold cross-validation, for example, may be determined based on the five-fold cross-validation.
  • the second sample image can be divided into five parts, four of which are used as the training set, and the other part is used as the verification set, and then multiple training and verifications are performed in other combinations. After different combinations are determined, the second initial model is trained and verified in different combinations at the same time, so that by training and verifying multiple combinations of sample data, the second initial model traverses all sample data, The trained model has better versatility and more accurate segmentation results.
  • the cross-validation process is mainly to verify the trained model through the verification data every time a certain number of iterative processes are performed. If the segmentation error meets the target condition, it can be stopped.
  • the embodiments of the present application will not go into details here.
  • the above steps 203 to 207 are to train the first image segmentation module and the second image segmentation module in the second initial model based on the plurality of second sample images, and stop when the number of first iteration stops is reached to obtain the first A process of module parameters of an image segmentation module and a second image segmentation module, in which the module parameters of the first image segmentation module are adjusted based on the first segmentation error in each iteration process, the second image segmentation The module parameters of the module are adjusted based on the second segmentation error during the second iteration.
  • Each time the second initial model executes this step 203 to step 207 is an iterative process.
  • the second initial model can perform the above process multiple times. Through multiple iterations, the module parameters of the two image segmentation modules are adjusted, that is, The process of training the first image segmentation module and the second image segmentation module separately.
  • the computer device when adjusting the module parameters of the two image segmentation modules, may also adjust the module parameters of the modal fusion module, so that modal fusion is trained in this training process The module parameters of the module.
  • both the first image segmentation module and the second image segmentation module may be convolutional neural network models.
  • the model can calculate the error of the predicted result and propagate it back to the volume.
  • the product neural network model it is possible to solve the convolution template parameter w and the bias parameter b of the neural network model through a gradient descent algorithm.
  • the second initial model in the computer device divides the plurality of second sample images based on the trained first image segmentation module and the second image segmentation module to obtain the first segmentation result and the first Two split results.
  • the first image segmentation module and the second image segmentation module are suitable for segmenting 3D images and 2D images, respectively, for a second sample image
  • the first image segmentation module may segment the second sample image more accurately
  • the second image segmentation module is very inaccurate for the segmentation result of the second sample image, so if the second initial model directly uses the combined result of the two modules, the final segmentation result that may be obtained is subject to the segmentation of the second image segmentation module The influence of the result leads to a decrease in the accuracy of the final segmentation result.
  • the second initial model can also be based on the training of the two modules, training a mixed strategy of the two modules, that is, training for a second sample image, which one to choose Module or two modules are better for segmenting the second sample image.
  • the second initial model can use the two modules completed by training to segment the second sample image to obtain the first segmentation result and the second segmentation result, and evaluate the two segmentation results and the comprehensive segmentation result of the two segmentation results. To determine which segmentation result is more accurate.
  • the second initial model in the computer device obtains a fifth segmentation result based on the first segmentation result and the second segmentation result.
  • the fifth segmentation result is a comprehensive segmentation result of the first segmentation result and the second segmentation result.
  • the process of obtaining the fifth segmentation result by the second initial model may be: the second initial model uses the average value of the first segmentation result and the second segmentation result as the fifth segmentation result, that is, for each pixel For each type of probability, the average value of the probability in the first segmentation result and the probability in the second segmentation result can be used as the probability in the fifth segmentation result.
  • the first segmentation result and the second segmentation result may also correspond to weights.
  • the process of obtaining the fifth segmentation result by the second initial model may be: the second initial model compares the first segmentation result and the second segmentation result Perform weighted summation to obtain the fifth segmentation result.
  • the second initial model in the computer device obtains the first segmentation error and the second segmentation based on the label of the second sample image, the first segmentation result, the second segmentation result, and the fifth segmentation result of each second sample image Error and third segmentation error.
  • the third segmentation error is the segmentation error of the fifth segmentation result.
  • the second initial model may determine the segmentation error of each segmentation result based on the label of the second sample image to determine whether each segmentation result is accurate.
  • the segmentation error of each segmentation result can also be obtained by using the first loss function or the second loss function, and the embodiments of the present application will not repeat them here.
  • the second initial model in the computer device adjusts the module selection parameters in the second initial model based on the first segmentation error, the segmentation error of the second segmentation result, and the third segmentation error until the second Stop when the number of iterations stops, and get the image segmentation model.
  • the module selection parameter is used to decide and select at least one of the first image segmentation module and the second image segmentation module to segment the first image.
  • the second initial model adjusts the module selection parameters based on the segmentation error of each segmentation result.
  • the resulting image segmentation model can make its own decision on how to select the module so that the segmentation result of the second sample image more acurrate.
  • the above steps 208 to 211 are based on the plurality of second sample images and the trained first image segmentation module and the second image segmentation module, training the module selection parameters in the second initial model until reaching the first It stops when the number of iterations stops, and the image segmentation model is obtained.
  • the module selection parameter is used to decide the process of segmenting the first image by at least one of the first image segmentation module and the second image segmentation module.
  • the process is The process of training the module selection parameters.
  • the module selection parameters are obtained by training based on the trained first image segmentation module, second image segmentation module, and the plurality of second sample images.
  • the second iteration stop count may also be determined based on cross-validation.
  • the second iteration stopping times may also be determined based on the k-fold cross-validation method, for example, may be determined based on the five-fold cross-validation method, and the embodiments of the present application will not repeat them here.
  • the model parameters of the second initial model include module parameters of the first image segmentation module, module parameters of the second image segmentation module, module parameters and module selection parameters of the modal fusion module in the second initial model . Then, the above steps 202 to 211 are based on a plurality of second sample images, training the second initial model to obtain an image segmentation model.
  • the segmentation result may be further post-processed to obtain a final divided image. That is, the second initial model may obtain a third image corresponding to the second sample image based on at least one of the first segmentation result and the second segmentation result, and thus determine the final output based on the third image corresponding to the second sample image
  • the second image corresponding to the two sample images is the segmented image corresponding to the second sample image.
  • the third image corresponding to the second sample image may be an image corresponding to the first segmentation result, an image corresponding to the second segmentation result, or an average of the first segmentation result and the second segmentation result Or the image obtained after weighted summation.
  • the post-processing process may be performed based on the distribution information of the target area in the image information acquired in step 203 above.
  • the second initial model may be based on the multiple target regions in the third image corresponding to the second sample image and the distribution information of the multiple target regions indicated by the image information.
  • the three images are post-processed to obtain a second image corresponding to the second sample image, the target area is the area where the pixels of the target type are located in the third image corresponding to the second sample image, and the second image corresponding to the second sample image
  • the distribution type, the number of target regions, and the size range of the target regions in the multiple target regions are the same as the distribution information of the multiple target regions.
  • the post-processing process may include any one or more of the following steps: when the number or size range of the target area in the third image corresponding to the second sample image and the number of target areas indicated by the image information
  • the second initial model filters out the part of the third image corresponding to the second sample image that does not meet the number or size range of the multiple target areas; or, when any target area exists
  • the second initial model changes the background pixels to pixels of the target type corresponding to the target area.
  • the distribution type of the first target area and the second target area is determined to be completely nested according to the distribution information of the target area, that is, the second target area should be inside the first target area, if If there is a second target area outside the first target area in the third image corresponding to the second sample image, the second target area outside the first target area can be filtered out. As shown in FIG. 7, if the distribution type of the first target area and the second target area is determined to be completely nested according to the distribution information of the target area, that is, the second target area should be inside the first target area, if If there is a second target area outside the first target area in the third image corresponding to the second sample image, the second target area outside the first target area can be filtered out. As shown in FIG.
  • the second target area should be outside the first target area, if the second sample If the third image corresponding to the image has a second target area inside the first target area, the second target area inside the first target area may be filled as the first target area.
  • the pixels in the target area should be foreground pixels. If there are background pixels in the target area in the third image corresponding to the second sample image, the background pixels can be corrected. For example, taking human tissue as an example, there should be no voids in the human tissue part.
  • the part can be filled to correct the segmentation result.
  • steps may also be included, and the embodiments of the present application are not enumerated here one by one.
  • the image segmentation model training is completed, and the image segmentation model acquires the image information of the multiple second sample images during the training process.
  • the computer device can call the image A segmentation model.
  • the image segmentation model splits the first image according to the image information and outputs a second image. Specifically, how the image segmentation model splits the first image specifically, as shown in FIG. 10 below Embodiments, and the segmentation process is the same as some steps in the image segmentation model training process, and embodiments of the present application will not repeat them here.
  • the image segmentation model includes a 3D network (Net) and a 2D network (Net), that is, a first image segmentation module and a second image segmentation Module, among them, 3D Net can adopt 2-stage flow design, after coarse prediction, then fine prediction, that is, after coarse division, then fine division.
  • 3D Net can adopt 2-stage flow design, after coarse prediction, then fine prediction, that is, after coarse division, then fine division.
  • For the input sample data you can input the sample data into 3D Net and 2D Net, and after obtaining the probability maps through the two networks, you can use different hybrid strategies to fuse the probability maps, that is, you can train the module to select parameters and determine to choose a single Network or choose two networks.
  • the image segmentation model can first preprocess the sample data. After the two networks are segmented, the image segmentation model can also post-process the results to obtain the final output segmented image.
  • the image segmentation model provided by the embodiments of the present application is highly versatile, and when applied to medical image segmentation, it is scene-specific to medical images, that is, it is scene-specific to human tissue images, and the model can be automatically trained, the user only needs to provide The data can be automatically trained without manual participation in parameter adjustment. Moreover, the image segmentation model provided by the embodiment of the present application has been verified in 10 different medical scenes, and all have good segmentation effects. And the image segmentation model can be automatically extended to other medical application scenarios, and has strong universality of medical images.
  • the embodiment of the present application pre-trains the initial model with multiple human tissue images, so that the initial model has a priori knowledge about human tissue, and when a certain human tissue image needs to be segmented, it is directly based on this human tissue image It is sufficient to train the pre-trained model without manually analyzing this human tissue image, and then redesign the model based on the analysis results, which effectively improves the versatility, applicability, and practicality of the image segmentation model.
  • the image segmentation model includes a first image segmentation module and a second image segmentation module, which can accurately segment both three-dimensional images and two-dimensional images, further improving the versatility, applicability and practicability of the image segmentation model It also improves the segmentation accuracy of the image segmentation model.
  • FIG. 10 is a flowchart of an image segmentation method provided by an embodiment of the present application.
  • the image segmentation method is applied to a computer device, and the computer device may be the computer device 101 in the foregoing implementation environment.
  • the image segmentation model is mainly called when the first image to be segmented is acquired, and the image segmentation model splits the first image according to the image information and outputs the second image. Detailed description has been made.
  • the image segmentation model includes only the first image segmentation module and the second image segmentation module as an example for description. Referring to FIG. 10, the image segmentation method may include the following steps:
  • the computer device acquires the first image to be divided.
  • the computer device executes this step 1001 when it detects an image segmentation operation, and can also receive the first image to be segmented imported by the user, and can also receive an image segmentation request sent by another computer device, where the image segmentation request carries the first segment to be segmented Image, the first image to be divided is extracted from the image segmentation request, or the image segmentation request may carry relevant information of the first image, and the computer device may perform step 1001 based on the relevant information.
  • the computer The device can also acquire the first image to be segmented through the imaging principle.
  • the embodiment of the present application does not limit the specific acquisition method and acquisition timing of the first image to be divided.
  • another computer device may acquire the first image to be divided by imaging principles, and send the first image to be divided to the computer device, and the computer device obtains the first image to be divided, the first image may
  • the following steps may be performed, and the first image is segmented using an image segmentation model obtained by training from the sample image of the target human tissue.
  • the computer device invokes the image segmentation model.
  • the image segmentation model includes a first image segmentation module and a second image segmentation module.
  • the first image segmentation module and the second image segmentation module respectively correspond to a segmentation algorithm.
  • the first image segmentation module is used to segment a three-dimensional image
  • the second image segmentation module is used to segment a two-dimensional image.
  • An image segmentation model may be pre-stored in the computer device.
  • the computer device is the computer device shown in FIG. 2, that is, the image segmentation model stored on the computer device is trained on the computer device .
  • the computer device is not the computer device shown in FIG. 2, that is, the image segmentation model can be trained on other computer devices, and the computer device can obtain the trained image segmentation model from other computer devices .
  • the image segmentation model can be called from other computer devices in real time. The application examples do not limit this.
  • the computer device inputs the first image into the image segmentation model, and the image segmentation model acquires attribute information of the first image.
  • the image segmentation model can obtain the attribute information of the first image.
  • the image segmentation model is a trained model instead of the second initial model, and when the image segmentation model is used ,
  • the attribute information of the first image can be obtained without obtaining the number of images, the distribution information of the target area, and so on.
  • the attribute information may include a gray scale range, a number of modes, and a size range, which are not limited in the embodiments of the present application.
  • the image segmentation model in the computer device preprocesses the first image according to the attribute information of the first image.
  • the pre-processing process of the first image by the image segmentation model may also include any one or more of the following steps:
  • Step 1 When it is determined that there is an abnormal pixel in the first image based on the attribute information, the image segmentation model deletes the abnormal pixel.
  • Step 2 When it is determined according to the attribute information that the gray range of the first image after deleting abnormal pixels is greater than the target range, the image segmentation model normalizes the first image and adjusts the gray range of the first image to Within target range.
  • Step 3 When it is determined that the number of channels of the first image is greater than one according to the attribute information, the image segmentation model subtracts the average value of the target image from each pixel value of the first image.
  • Step 4 When it is determined that the number of modalities of the first image is greater than one according to the attribute information, the image segmentation model inputs the first image into a modal fusion module, and the modal fusion module selects multiple pixel values from the first image Screening is performed to obtain the target number of pixel values of the pre-processed first image, and the number of modalities of the pre-processed first image is one.
  • Steps 1 to 4 in step 1004 are the same as steps 1 to 4 in step 204 above, and embodiments of the present application will not repeat them here.
  • the image segmentation model in the computer device inputs the preprocessed first image into at least one of the first image segmentation module and the second image segmentation module, and the first image segmentation module and the second image segmentation module At least one module divides the first image to obtain a third image.
  • the image segmentation model may select the first image segmentation module based on the module selection parameter And at least one of the second image segmentation modules to segment the first image.
  • this step 1005 may include the following three possible situations:
  • the image segmentation model is based on the module selection parameters of the image segmentation model, the first image segmentation module segments the first image to obtain a first segmentation result, and based on the first segmentation result, a third image is obtained ,
  • the first segmentation result is used to indicate the probability that each pixel of the first image is each of at least two types.
  • the image segmentation model is based on the module selection parameters of the image segmentation model, the first image is segmented by the second image segmentation module to obtain a second segmentation result, and based on the second segmentation result, a third image is obtained ,
  • the second segmentation result is used to indicate the probability that each pixel of the first image is each of at least two types.
  • the image segmentation model is based on the module selection parameters of the image segmentation model, and the first image segmentation module and the second image segmentation module respectively segment the first image to obtain the first segmentation result and the second segmentation result Based on the first segmentation result and the second segmentation result, a third image is obtained.
  • the process of obtaining the third image is also the same as the content in the above step 211.
  • the above three cases respectively correspond to the three acquisition processes of the third image, which are:
  • the third image is an image corresponding to the first segmentation result
  • the third image is an image corresponding to the second segmentation result.
  • the third image is obtained by averaging or weighting the first segmentation result and the second segmentation result.
  • the first image segmentation module may divide the first image into the following steps: the first image segmentation module performs two operations on the first image based on the module parameters of the first image segmentation module Sub-classification to obtain the first segmentation result.
  • the first classification object in the two classifications is all pixels of the first image, and the second classification object is the foreground pixels in the first classification result .
  • it may include the following steps one to three:
  • Step 1 The first image segmentation module classifies each pixel of the first image based on the module parameters of the first image segmentation module to obtain a third segmentation result, which is used to indicate the first image
  • Each pixel of is a probability of each type in at least two types.
  • the at least two types include a foreground and a background, and the foreground is any type other than the background.
  • Step 2 The first image segmentation module classifies each foreground pixel in the third segmentation result based on the third segmentation result and the module parameters of the first image segmentation module to obtain a fourth segmentation result.
  • the segmentation result is used to indicate the probability that each foreground pixel in the third segmentation result is each of the at least two types.
  • Step 3 The first image segmentation module obtains a first segmentation result based on the third segmentation result and the fourth segmentation result.
  • the second image segmentation module's segmentation process of the first image may include the following steps one and two:
  • Step 1 The second image segmentation module performs feature extraction on the first image based on the module parameters of the second image segmentation module.
  • Step 2 The second image segmentation module classifies each pixel of the first image based on the extracted features to obtain a second segmentation result.
  • the image segmentation model may After processing, a plurality of first sub-images are obtained, and the first sub-images are two-dimensional images.
  • the segmentation process of the first image by the second image segmentation module includes: based on the module parameters of the second image segmentation module, the second image segmentation module separately segments the first sub-images corresponding to the first image To obtain multiple second sub-segmentation results; the second image segmentation module fuse the multiple sub-segmentation results to obtain a second segmentation result.
  • the image segmentation model in the computer device performs post-processing on the third image according to the image information of the plurality of second sample images in the image segmentation model to output the second image.
  • the image segmentation model can also post-process the third image.
  • the post-processing process can also be: the image segmentation model is based on multiple target regions in the third image And the distribution information of the plurality of target areas indicated by the image information, post-processing the third image to obtain a second image, the target area is the area where the pixels of the target type in the third image are located, and the second The distribution type, the number of target regions, and the size range of the target regions in the image are the same as the distribution information of the multiple target regions.
  • the image segmentation model may also perform any one or more of the following steps: when the number or size range of the target area in the third image is indicated by the image information When the number or size range of the plurality of target areas is different, the image segmentation model filters out the portion of the third image that does not conform to the number or size range of the plurality of target areas; or, when any target area exists For background pixels, change the background pixels to pixels of the target type corresponding to the target area.
  • the above steps 1003 to 1006 are the process of segmenting the first image and outputting the second image based on at least one of the first image segmentation module and the second image segmentation module in the image segmentation model and the image information,
  • the computer device can store the second image, of course, the first image and the second image can also be stored correspondingly, if the computer device is the above image segmentation process based on the image segmentation request of other computer devices , You can also send the second image to the other computer device.
  • the image segmentation model includes only the first image segmentation module and the second image segmentation module as examples.
  • the image segmentation model may further include only one image segmentation module or more
  • the image segmentation process is the same as the above process, so I wo n’t repeat them here.
  • the embodiment of the present application pre-trains the initial model with multiple human tissue images, so that the initial model has a priori knowledge about human tissue, and when a certain human tissue image needs to be segmented, it is directly based on this human tissue image
  • the pre-trained model can be trained without manual analysis of this human tissue image, and then the model is redesigned based on the analysis results, and the image segmentation model obtained by the above method can accurately segment this human tissue image , Effectively improve the versatility, applicability and practicality of the image segmentation method, and also effectively improve the accuracy of the image segmentation method.
  • steps in the embodiments of the present application are not necessarily executed in the order indicated by the step numbers. Unless clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. The order is not necessarily sequential, but may be executed in turn or alternately with at least a part of other steps or sub-steps or stages of other steps.
  • a computer device in one embodiment, includes an image segmentation device.
  • the image segmentation device includes various modules, and each module may be implemented in whole or in part by software, hardware, or a combination thereof.
  • FIG. 11 is a schematic structural diagram of an image segmentation device provided by an embodiment of the present application. Referring to FIG. 11, the device includes:
  • the training module 1101 is used to pre-train the first initial model based on multiple first sample images to obtain a second initial model.
  • the multiple first sample images include multiple human tissue images, and the second initial model Includes distribution information of multiple target areas corresponding to the multiple human tissues;
  • the training module 1101 is also used to train the second initial model based on multiple second sample images to obtain an image segmentation model.
  • the multiple second sample images are target human tissue images.
  • the image segmentation model during training Acquiring image information of the plurality of second sample images, the image information of the plurality of second sample images at least includes distribution information of a plurality of target regions corresponding to the target human tissue;
  • the segmentation module 1102 is configured to call the image segmentation model when the first image to be segmented is acquired, and the image segmentation model segments the first image according to the image information and outputs the second image.
  • the first initial model, the second initial model, and the image segmentation model all include a first image segmentation module and a second image segmentation module, where the first image segmentation module and the second image segmentation module correspond to In a segmentation algorithm, the first image segmentation module is used to segment a three-dimensional image, and the second image segmentation module is used to segment a two-dimensional image;
  • the segmentation module 1102 is used to segment the first image and output the second image based on at least one of the first image segmentation module and the second image segmentation module in the image segmentation model and the image information.
  • the segmentation module 1102 is used to:
  • the third image is post-processed to output the second image.
  • the segmentation module 1102 is used to:
  • the abnormal pixel is deleted
  • the first image is normalized, and the gray range of the first image is adjusted to be within the target range;
  • each pixel value of the first image is subtracted from the average value of the target image
  • the first image is input to a modal fusion module, and the modal fusion module selects from multiple pixel values of the first image to obtain preprocessing After the target number of pixel values of the first image, the pre-processed first image has a modal number of one.
  • the segmentation module 1102 is used to post-process the third image based on the multiple target regions in the third image and the distribution information of the multiple target regions indicated by the image information, to obtain the first Two images
  • the target area is the area where the pixels of the target type in the third image are located, the distribution type of the multiple target areas, the number of target areas, the size range of the target area and the size of the multiple target areas in the second image
  • the distribution information is the same.
  • the segmentation module 1102 is used to:
  • the number or size range of target areas in the third image is different from the number or size range of the plurality of target areas indicated by the image information, the number or size of the plurality of target areas in the third image does not match Filter out parts of different ranges; or,
  • the segmentation module 1102 is used to:
  • the first image segmentation module segments the first image to obtain a first segmentation result, and based on the first segmentation result, a third image is obtained.
  • the first segmentation result is For indicating the probability that each pixel of the first image is at least two types of each type; or,
  • the second image segmentation module segments the first image to obtain a second segmentation result, and based on the second segmentation result, a third image is obtained. For indicating the probability that each pixel of the first image is at least two types of each type; or,
  • the first image segmentation module and the second image segmentation module respectively segment the first image to obtain a first segmentation result and a second segmentation result. Based on the first segmentation result and As a result of the second segmentation, a third image is obtained.
  • the segmentation module 1102 is used to:
  • the first image is classified twice to obtain a first segmentation result.
  • the first classification object in the two classifications is all pixels of the first image.
  • the classification object of the second classification is the foreground pixel in the first classification result;
  • the segmentation module 1102 is used to:
  • each pixel of the first image is classified to obtain a second segmentation result.
  • the training module 1101 is used to:
  • the image information further includes attribute information of each second sample image
  • the training module 1101 is also used to preprocess each second sample image based on the attribute information of each second sample image, and input the preprocessed multiple second sample images into the first image segmentation Module and the second image segmentation module.
  • the module parameters of the first image segmentation module are adjusted based on the first segmentation error during each iteration, and the first segmentation error is the segmentation of the first segmentation result corresponding to the first image segmentation module Error, the first segmentation error acquisition process is implemented using a first loss function, and the weight of each type of pixel in the first loss function is based on the type of pixel in the image information of the plurality of second sample images. The proportion of the second sample images is determined;
  • the module parameters of the second image segmentation module are adjusted based on the second segmentation error in the second iteration process, the second segmentation error is the segmentation error of the second segmentation result corresponding to the second image segmentation module, and the second segmentation error
  • the acquisition process is implemented using a second loss function, and the weight of the second loss function is determined based on the online hard sample mining OHEM algorithm;
  • the first iteration stop number and the second iteration stop number are determined based on the cross-validation.
  • the image information further includes the number of images of the plurality of second sample images
  • the training module 1101 is also used to obtain an image segmentation sub-module corresponding to the number of images based on the number of images as the second image segmentation module for training, the second image segmentation module includes at least one image segmentation sub-module, Different image segmentation sub-modules have different depths.
  • the device provided in the embodiment of the present application pre-trains the initial model with multiple human tissue images, so that the initial model has a priori knowledge about human tissue.
  • a certain human tissue image needs to be segmented, it is directly based on this Human body image can be used to train the pre-trained model without the need to manually analyze this human tissue image, and then redesign the model based on the analysis results, and the image segmentation model obtained by the above method can be applied to this human tissue
  • the accurate image segmentation effectively improves the versatility, applicability and practicality of the image segmentation method, and also effectively improves the accuracy of the image segmentation method.
  • the image segmentation device provided in the above embodiments only uses the above-mentioned division of each functional module as an example when dividing an image.
  • the above-mentioned functions can be allocated by different functional modules according to needs.
  • the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.
  • the image segmentation apparatus and the image segmentation method embodiment provided in the above embodiments belong to the same concept. For the specific implementation process, refer to the method embodiment, and details are not described here.
  • the above computer device may be provided as a terminal shown in FIG. 12 below, or may be provided as a server shown in FIG. 13 below, which is not limited in the embodiment of the present application.
  • the terminal 1200 may be: a smartphone, a tablet computer, an MP3 player (Moving Pictures Experts Group Audio Audio Layer III, motion picture expert compression standard audio level 3), MP4 (Moving Pictures Experts Group Audio Audio Layer IV, motion picture expert compression standard audio Level 4) Player, laptop or desktop computer.
  • the terminal 1200 may also be called other names such as user equipment, portable terminal, laptop terminal, and desktop terminal.
  • the terminal 1200 includes a processor 1201 and a memory 1202.
  • the processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on.
  • the processor 1201 may adopt at least one hardware form of DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). achieve.
  • the processor 1201 may also include a main processor and a coprocessor.
  • the main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor for processing data in the standby state.
  • the processor 1201 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used to render and draw content that needs to be displayed on the display screen.
  • the processor 1201 may also include an AI (Artificial Intelligence, Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, Artificial Intelligence
  • the memory 1202 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 1202 may also include high-speed random access memory, and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction that is executed by the processor 1201 to implement the image segmentation provided by the method embodiment in the present application Method or image segmentation model training method.
  • the terminal 1200 may optionally include a peripheral device interface 1203 and at least one peripheral device.
  • the processor 1201, the memory 1202, and the peripheral device interface 1203 may be connected by a bus or a signal line.
  • Each peripheral device may be connected to the peripheral device interface 1203 through a bus, a signal line, or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 1204, a touch display 1205, a camera 1206, an audio circuit 1207, a positioning component 1208, and a power supply 1209.
  • the peripheral device interface 1203 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1201 and the memory 1202.
  • the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1201, the memory 1202, and the peripheral device interface 1203 or Both can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the radio frequency circuit 1204 is used to receive and transmit RF (Radio Frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 1204 communicates with the communication network and other communication devices through electromagnetic signals.
  • the radio frequency circuit 1204 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal.
  • the radio frequency circuit 1204 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on.
  • the radio frequency circuit 1204 can communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity, wireless fidelity) networks.
  • the radio frequency circuit 1204 may further include a circuit related to NFC (Near Field Communication), which is not limited in this application.
  • the display screen 1205 is used to display a UI (User Interface).
  • the UI may include graphics, text, icons, video, and any combination thereof.
  • the display screen 1205 also has the ability to collect touch signals on or above the surface of the display screen 1205.
  • the touch signal may be input to the processor 1201 as a control signal for processing.
  • the display screen 1205 can also be used to provide virtual buttons and / or virtual keyboards, also called soft buttons and / or soft keyboards.
  • the display screen 1205 may be a flexible display screen, which is disposed on the curved surface or the folding surface of the terminal 1200. Even, the display screen 1205 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen.
  • the display screen 1205 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light emitting diode) and other materials.
  • the camera assembly 1206 is used to collect images or videos.
  • the camera assembly 1206 includes a front camera and a rear camera.
  • the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal.
  • there are at least two rear cameras which are respectively one of the main camera, the depth-of-field camera, the wide-angle camera, and the telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera Integrate with wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting function or other fusion shooting functions.
  • the camera assembly 1206 may also include a flash.
  • the flash can be a single-color flash or a dual-color flash. Dual color temperature flash refers to the combination of warm flash and cold flash, which can be used for light compensation at different color temperatures.
  • the audio circuit 1207 may include a microphone and a speaker.
  • the microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1201 for processing, or input them to the radio frequency circuit 1204 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively installed in different parts of the terminal 1200.
  • the microphone can also be an array microphone or an omnidirectional acquisition microphone.
  • the speaker is used to convert the electrical signal from the processor 1201 or the radio frequency circuit 1204 into sound waves.
  • the speaker can be a traditional thin-film speaker or a piezoelectric ceramic speaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible by humans, but also convert electrical signals into sound waves inaudible to humans for ranging and other purposes.
  • the audio circuit 1207 may further include a headphone jack.
  • the positioning component 1208 is used to locate the current geographic location of the terminal 1200 to implement navigation or LBS (Location Based Service, location-based service).
  • the positioning component 1208 may be a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China, the Grenas system of Russia, or the Galileo system of the European Union.
  • GPS Global Positioning System
  • the power supply 1209 is used to supply power to various components in the terminal 1200.
  • the power source 1209 may be alternating current, direct current, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 1200 further includes one or more sensors 1210.
  • the one or more sensors 1210 include, but are not limited to, an acceleration sensor 1211, a gyro sensor 1212, a pressure sensor 1213, a fingerprint sensor 1214, an optical sensor 1215, and a proximity sensor 1216.
  • the acceleration sensor 1211 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established with the terminal 1200.
  • the acceleration sensor 1211 can be used to detect the components of gravity acceleration on three coordinate axes.
  • the processor 1201 may control the touch display 1205 to display the user interface in a landscape view or a portrait view according to the gravity acceleration signal collected by the acceleration sensor 1211.
  • the acceleration sensor 1211 can also be used for game or user movement data collection.
  • the gyro sensor 1212 can detect the body direction and rotation angle of the terminal 1200, and the gyro sensor 1212 can cooperate with the acceleration sensor 1211 to collect a 3D action of the user on the terminal 1200.
  • the processor 1201 can realize the following functions according to the data collected by the gyro sensor 1212: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 1213 may be disposed on a side frame of the terminal 1200 and / or a lower layer of the touch display 1205.
  • the pressure sensor 1213 can detect the user's grip signal on the terminal 1200, and the processor 1201 can perform left-right hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 1213.
  • the processor 1201 controls the operability control on the UI interface according to the user's pressure operation on the touch display 1205.
  • the operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 1214 is used to collect the user's fingerprint, and the processor 1201 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 1214, or the fingerprint sensor 1214 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 1201 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 1214 may be provided on the front, back, or side of the terminal 1200. When a physical button or manufacturer logo is provided on the terminal 1200, the fingerprint sensor 1214 may be integrated with the physical button or manufacturer logo.
  • the optical sensor 1215 is used to collect ambient light intensity.
  • the processor 1201 may control the display brightness of the touch display 1205 according to the ambient light intensity collected by the optical sensor 1215. Specifically, when the ambient light intensity is high, the display brightness of the touch display 1205 is increased; when the ambient light intensity is low, the display brightness of the touch display 1205 is decreased.
  • the processor 1201 may also dynamically adjust the shooting parameters of the camera assembly 1206 according to the ambient light intensity collected by the optical sensor 1215.
  • the proximity sensor 1216 also called a distance sensor, is usually provided on the front panel of the terminal 1200.
  • the proximity sensor 1216 is used to collect the distance between the user and the front of the terminal 1200.
  • the processor 1201 controls the touch display 1205 to switch from the bright screen state to the breathing state; when the proximity sensor 1216 detects When the distance from the user to the front of the terminal 1200 gradually becomes larger, the processor 1201 controls the touch display 1205 to switch from the screen-on state to the screen-on state.
  • FIG. 13 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 1300 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units (CPU) 1301 and one Or more than one memory 1302, wherein at least one instruction is stored in the memory 1302, and the at least one instruction is loaded and executed by the processor 1301 to implement the image segmentation method or image segmentation model training method provided by each of the above method embodiments .
  • the server may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output.
  • the server may also include other components for implementing device functions, which will not be repeated here.
  • the above computer device may be provided as a server shown in FIG. 14 below, or may be provided as a terminal shown in FIG. 15 below, which is not limited in the embodiment of the present application.
  • the above-mentioned computer device may be provided as a server shown in FIG. 14 described below.
  • the server includes a processor, memory, network interface, and database connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, a computer program, and a database.
  • the internal memory provides an environment for the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer device is used to store image data.
  • the network interface of the computer device is used to communicate with external terminals through a network connection.
  • the computer program is executed by the processor to implement an image segmentation method or image segmentation model training method.
  • the above-mentioned computer device may be provided as a terminal shown in FIG. 15 described below.
  • the terminal includes a processor, memory, network interface, display screen, and input device connected through a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and computer programs.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the network interface of the computer device is used to communicate with external terminals through a network connection.
  • the computer program is executed by the processor to implement an image segmentation method or image segmentation model training method.
  • the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or may be a button, a trackball or a touchpad provided on the computer device housing , Can also be an external keyboard, touchpad or mouse.
  • FIGS. 14 and 15 are only block diagrams of some structures related to the solution of the present application, and do not constitute a limitation on the server and terminal to which the solution of the present application is applied. 2.
  • the terminal may include more or less components than shown in the figure, or combine certain components, or have different component arrangements.
  • the image segmentation apparatus provided by the present application may be implemented in the form of a computer-readable instruction, and the computer-readable instruction may be run on the server shown in FIG. 14 or may be shown in FIG. 15 Run on the terminal.
  • the memory of the server or the terminal may store various program modules constituting the image segmentation device, such as the training module 1101 and the segmentation module 1102.
  • the computer-readable instructions formed by the various program modules cause the processor to execute the steps in the image segmentation method or the image segmentation model training method of the various embodiments of the present application described in this specification.
  • An embodiment of the present application provides a computer-readable storage medium that stores computer-readable instructions, which are loaded by a processor and have an image segmentation method or image segmentation model to implement the above-described embodiments Operations in training methods.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous chain (Synchlink) DRAM
  • SLDRAM synchronous chain (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种图像分割方法、装置、计算机设备及存储介质，属于计算机技术领域。方法包括：基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型，第二初始模型中包括多种人体组织对应的多个目标区域的分布信息；基于多个第二样本图像，对第二初始模型进行训练，得到图像分割模型；当获取到待分割的第一图像时，调用图像分割模型，由图像分割模型根据第二样本图像的图像信息，对第一图像进行分割，输出第二图像。

Description

图像分割方法、装置、计算机设备及存储介质
本申请要求于2018年10月16日提交中国专利局，申请号为201811205146.4，申请名称为“图像分割方法、装置、计算机设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种图像分割方法、装置、计算机设备及存储介质。
背景技术
随着计算机技术的发展，图像分割技术应用越来越广泛，例如，医学图像分割、自然图像分割等。其中，图像分割技术是指把图像分成若干个特定的、具有独特性质的区域并提出感兴趣目标的技术。例如，人体组织图像分割场景中，可以对医学图像进行分割，使得分割后的图像中能明显区分人体各个组织。
目前，图像分割方法通常采用级联三维全卷积网络(Cascaded Three-dimensional fully convolutional networks，Cascaded 3D FCN)和金字塔场景解析网络(Pyramid Scene Parsing Network，Pspnet)两种模型实现。其中，Cascaded3D FCN为三维(Three-dimensional，3D)网络，主要用于对腹部组织进行分割。Pspnet为二维(Two-dimensional，2D)网络，主要用于对自然图像进行分割。上述图像分割方法通常需要技术人员针对图像分割需求，对某种人体组织图像进行分析，分析图像中待分割的是哪种人体组织，这种人体组织的图像的像素点分布有什么特性，并根据分析结果设计模型，从而获取这种人体组织的样本图像对设计好的模型进行训练，从而可以使用训练得到的模型对需要分割的图像进行分割。
上述图像分割方法中均需技术人员对图像进行分析，并对模型进行设计，如果需要对其他人体组织图像进行分割，则需要技术人员对其他人体组织图像进行分析，来重新设计模型，而无法在之前设计好的模型的基础上直接进行训练，因此，上述图像分割方法的通用性、适用性和实用性差。
发明内容
本申请提供的各种实施例，提供了一种图像分割方法、装置、计算机设备及存储介质。所述技术方案如下：
一方面，提供了一种图像分割方法，所述方法包括：
计算机设备基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型，所述多个第一样本图像包括多种人体组织图像，所述第二初始模型中包括所述多种人体组织对应的多个目标区域的分布信息；
所述计算机设备基于多个第二样本图像，对所述第二初始模型进行训练，得到图像分割模型，所述多个第二样本图像为目标人体组织图像，训练过程中所述图像分割模型获取得到所述多个第二样本图像的图像信息，所述多个第二样本图像的图像信息至少包括所述目标人体组织对应的多个目标区域的分布信息；
当获取到待分割的第一图像时，所述计算机设备调用所述图像分割模型，由所述图像分割模型根据所述图像信息，对所述第一图像进行分割，输出第二图像。
一方面，提供了一种图像分割装置，所述装置包括：
训练模块，用于基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型，所述多个第一样本图像包括多种人体组织图像，所述第二初始模型中包括所述多种人体组织对应的多个目标区域的分布信息；
所述训练模块，还用于基于多个第二样本图像，对所述第二初始模型进行训练，得到图像分割模型，所述多个第二样本图像为目标人体组织图像，训练过程中所述图像分割模型获取得到所述多个第二样本图像的图像信息，所述多个第二样本图像的图像信息至少包括所述目标人体组织对应的多个目标区域的分布信息；
分割模块，用于当获取到待分割的第一图像时，调用所述图像分割模型，由所述图像分割模型根据所述图像信息，对所述第一图像进行分割，输出第二图像。
一方面，提供了一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时, 使得所述处理器执行上述实施例所述的方法。
一方面，提供了一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述实施例所述的方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其他特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种图像分割方法的实施环境；
图2是本申请实施例提供的一种图像分割模型训练方法的流程图；
图3是本申请实施例提供的一种模态融合模块的处理流程示意图；
图4是本申请实施例提供的一种图像采样方式示意图；
图5是本申请实施例提供的一种图像采样方式示意图；
图6是本申请实施例提供的一种3D模型的结构示意图；
图7是本申请实施例提供的一种图像后处理方式示意图；
图8是本申请实施例提供的一种图像后处理方式示意图；
图9是本申请实施例提供的一种图像分割模型的架构示意图；
图10是本申请实施例提供的一种图像分割方法的流程图；
图11是本申请实施例提供的一种图像分割装置的结构示意图；
图12是本申请实施例提供的一种终端的结构示意图；
图13是本申请实施例提供的一种服务器的结构示意图；
图14是本申请实施例提供的一种服务器的结构示意图；
图15是本申请实施例提供的一种终端的结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图1是本申请实施例提供的一种图像分割方法的实施环境，参见图1，该实施环境中可以包括多个计算机设备。其中，该多个计算机设备可以通过有线连接方式实现数据交互，也可以通过无线网络连接方式实现数据交互，本申请实施例对此不作限定。
在本申请实施例中，计算机设备101可以用于对图像进行分割，在一些实施例中，该图像可以是医学图像，也就是人体组织图像，则本申请实施例所提供的图像分割方法可以应用于人体组织图像分割场景，例如，肝癌分割、脑癌及周边损伤分割、肺癌分割、胰脏癌症分割、大肠癌分割、肝脏入侵微血管分割、海马体结构分割、前列腺结构分割、左心房分割、胰脏分割、肝脏分割或脾脏分割等人体组织图像分割场景，当然也可以是其它人体组织图像分割场景。当然，该图像也可以是其它类型的图像，则该图像分割方法也可以应用于其他图像分割场景中，例如，风景图像分割场景。
该计算机设备102可以用于采集图像，并将采集到的图像发送至计算机设备101，由计算机设备101提供图像分割服务。在一些实施例中，该计算机设备101也可以采集图像，并对采集到的图像进行分割，本申请实施例对此不作限定。在一些实施例中，该计算机设备102也可以用于存储从其他计算机设备处获取到的图像，该计算机设备101可以从该计算机设备102中获取到存储的图像进行分割。
具体地，该计算机设备101和计算机设备102均可以被提供为终端，也可以被提供为服务器，本申请实施例对此不作限定。
图2是本申请实施例提供的一种图像分割模型训练方法的流程图，该图像分割模型训练方法可以应用于计算机设备中，该计算机设备可以为上述实施环境中的计算机设备101，也可以是其他计算机设备。也就是，可以在上述计算机设备101上训练得到图像分割模型，也可以在其他计算机设备上训练得到图像分割模型后，将该图像分割模型处理为配置文件，将该配置文件发送至计算机 设备101，则该计算机设备101中就存储有图像分割模型。当然，也可以由计算机设备101在有图像分割需求时，调用其它计算机设备上训练得到的图像分割模型，本申请实施例对此不作限定。参见图2，该方法可以包括以下步骤：
200、计算机设备基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型。
其中，该多个第一样本图像包括多种人体组织图像，例如，该多个第一样本图像可以包括肝部图像、脑部图像、肺部图像、胰脏图像、大肠图像、海马体部位的图像、前列腺部位的图像、心脏部位的图像和脾脏图像等人体组织图像，当然，还可以包括更多其他部位的图像，在此仅提供了一种示例性说明，而不对该多种人体组织图像的种类数量和具体种类进行限定。
在该步骤200中，计算机设备基于多种人体组织图像预训练得到了第二初始模型的模型参数，且该第二初始模型中包括该多种人体组织对应的多个目标区域的分布信息，也就是通过对多种人体组织图像进行分割，获取到了先验知识。这样通过不同场景的数据对模型进行预训练，可以使得该第二初始模型具备对人体组织图像进行分割的相关知识，可以用于模拟医学生在各个科室轮转，从而该医学生可以具备一定的医学知识或临床知识。
这样该第二初始模型已经具备一定的先验知识，后续需要对某种人体组织图像进行分割时，可以直接用这种人体组织的样本图像对第二初始模型进行训练即可，而无需由相关技术人员再对这种人体组织图像进行分析，重新设计模型，也就是，该第二初始模型具备一定的医学知识，各种人体组织图像均可以直接使用上述第二初始模型进行训练，可以有效提高该第二初始模型以及基于该第二初始模型训练得到的图像分割模型的实用性和通用性。
201、计算机设备获取多个第二样本图像，每个第二样本图像携带有标签，该标签用于指示第二样本图像的目标分割结果。
计算机设备可以基于多个第二样本图像训练得到图像分割模型，在一些实施例中，在该计算机设备中可以存储有该多个第二样本图像，在需要进行图像分割模型训练时，即可获取得到。需要说明的是，每个第二样本图像还可以携带用于指示目标分割结果的标签，其中，该目标分割结果是指该第二样本图像的正确的分割结果，或是指该第二样本图像的真实的分割结果。这样在模型训练过程中，可以获知训练中的模型对第二样本图像的分割是否准确，可以获知是否需要继续对模型进行训练，从而训练得到的模型对第二样本图像进行分割 时可以得到该目标分割结果，或者非常接近该目标分割结果。
在一些实施例中，该多个第二样本图像也可以存储于其他计算机设备，该计算机设备在需要进行图像分割模型训练时，可以从其他计算机设备处获取，本申请实施例对此不作限定。
例如，该多个第二样本图像可以存储于图像数据库中，每个第二样本图像还携带有标签。则该步骤201就可以为计算机设备从图像数据库中获取多个第二样本图像。
202、计算机设备将该多个第二样本图像输入该第二初始模型中。
在一些实施例中，上述步骤200之后，计算机设备可以直接对第二初始模型进行训练，也即是执行步骤201，再执行步骤202，相应地，该步骤202中，计算机设备获取多个第二样本图像后，可以基于该多个第二样本图像对第二初始模型进行训练，得到图像分割模型，以便于后续能够基于该图像分割模型对获取到的第一图像进行准确分割。在一些实施例中，上述步骤200之后，计算机设备还可以存储该第二初始模型，也可以将第二初始模型发送至其他计算机设备，由其他计算机设备基于第二初始模型，执行后续模型训练过程，则相应地，该步骤202中，计算机设备在获取到多个第二样本图像后，可以调用第二初始模型，将该多个第二样本图像输入该第二初始模型中，本申请实施例对具体采用哪种实现方式不作限定。
也就是，该步骤202中的计算机设备中可以存储有第二初始模型，在需要进行图像分割模型训练时，直接调用即可，或直接在步骤200之后，基于得到的第二初始模型进行训练，而无需调用。当然，该第二初始模型也可以存储于其他计算机设备，该计算机设备在需要进行图像分割模型训练时，可以从其他计算机设备处调用，本申请实施例对此不作限定。
需要说明的是，该第二初始模型的模型参数为初始值，计算机设备可以将该多个第二样本图像作为训练样本和验证样本，对第二初始模型进行训练，也就是通过第二样本图像对第二初始模型的模型参数进行调整，以使得多次调整后的模型参数能够在对第一图像进行分割时，得到的分割结果更准确。
具体地，计算机设备将该多个第二样本图像输入第二初始模型中，可以由第二初始模型对每个第二样本图像进行分割，基于第二初始模型的分割结果和第二样本图像的标签，也就是第二样本图像的目标分割结果，确定第二初始模型的分割能力，从而可以通过调整该第二初始模型的模型参数，以不断提高第 二初始模型的分割能力，以使得后续训练得到的图像分割模型能够准确分割。
203、计算机设备中的第二初始模型获取该多个第二样本图像的图像信息。
其中，该图像信息至少包括该目标人体组织对应的多个目标区域的分布信息。也就是，第二初始模型可以先获取多个第二样本图像的图像信息，从而获取到一定的先验知识，了解到所需要分割的目标人体组织对应的多个目标区域的分布信息，例如，该分布信息可以包括多个目标区域的分布类型、目标区域的数量和目标区域的大小范围，当然，也可以包括其他信息，本申请实施例在此不一一列举。
第二初始模型可以基于该多个第二样本图像的相关情况，初步获知后续模型训练完成后需要进行分割的第一图像的一些分割规律和处理规律。
具体地，第二初始模型可以基于该多个第二样本图像的标签中的信息，对该多个第二样本图像进行连通域处理，对连通域处理后的多个第二样本图像进行拓扑分析，得到目标人体组织对应的多个目标区域的分布信息。其中，该目标区域是指该多个第二样本图像中目标类型的像素点所在区域。
在一些实施例中，该图像信息还可以包括每个第二样本图像的属性信息，从而基于该第二样本图像的属性信息，对第二样本图像进行预处理从而使得该第二样本图像的一些基本信息更统一或更准确，从而对第二样本图像进行分割时更准确，当然，也可以对分割后的图像进行后处理，使得分割结果更准确。其中，该属性信息可以包括灰度范围、模态数量和尺寸范围等，本申请实施例对此不作限定。
在一些实施例中，上述第一初始模型和第二初始模型均可以包括第一图像分割模块和第二图像分割模块，该第一图像分割模块和第二图像分割模块分别对应于一种分割算法，该第一图像分割模块用于对三维(Three-dimensional，3D)图像进行分割，该第二图像分割模块用于对二维(Two-dimensional，2D)图像进行分割。相应地，基于该第二初始模型训练得到的图像分割模型也可以包括两个图像分割模块。这样得到的图像分割模型可以直接对三维图像进行分割，也可以将三维图像转化为二维图像进行分割，当然对于二维图像还可以直接进行分割，且通过两种分割算法，可以灵活选择不同的分割方式对图像进行分割，可以有效提高图像分割模型的实用性，也可以提高图像分割的准确性。
这样在该步骤203中，计算机设备获取到的该图像信息还可以包括该多个第二样本图像的图像数量，从而可以基于图像数量选择作为第二图像分割模块 的图像分割子模块，具体地，如何基于该图像数量，选择对第二样本图像的处理方式可以参见下述步骤205中第二图像分割模块的图像分割子模块的选择过程，本申请实施例在此先不作过多介绍。
需要说明的是，下述均以该步骤203中就获取到图像信息中包括该目标人体组织对应的多个目标区域的分布信息、每个第二样本图像的属性信息和该多个第二样本图像的图像数量，且第一初始模型、第二初始模型以及图像分割模型均包括第一图像分割模块和第二图像分割模块为例进行说明。
204、计算机设备中的第二初始模型根据该图像信息，对每个第二样本图像进行预处理，将预处理后的每个第二样本图像输入该第二初始模型中的第一图像分割模块和第二图像分割模块。
该步骤204是基于该每个第二样本图像的属性信息，对每个第二样本图像进行预处理，将预处理后的多个第二样本图像输入该第一图像分割模块和该第二图像分割模块中的过程，也就是该步骤204中预处理所依据的数据为图像信息中的每个第二样本图像的属性信息。
第二初始模型可以对第二样本图像进行预处理，使得预处理后的第二样本图像符合第一图像分割模块和第二图像分割模块的图像分割条件，也可以去除该第二样本图像中的异常像素点，或将该第一图像的像素点进行规范化等，从而预处理后的第二样本图像在图像分割时更准确。
在一些实施例中，第二样本图像的属性信息不同，该预处理过程也可以不同，具体地，该预处理过程可以包括以下任一个或多个步骤：
步骤一、当根据该属性信息确定该第二样本图像中存在异常像素点时，第二初始模型将该异常像素点删除。
在一些实施例中，异常像素点是指像素值为异常值的像素点，其中，异常像素点的像素值为该第二样本图像的多个像素值中与该多个像素值的平均值的偏差超过两倍标准差的像素值。另外，与平均值的偏差超过三倍标准差的像素值也可以为高度异常的异常像素点的像素值。如果该第二样本图像中存在异常像素点，可以将该第二样本图像中的异常像素点进行删除，以避免该异常像素点对分割结果造成影响，从而保证对该第二样本图像的分割结果更准确。
具体地，可以根据该第二样本图像的多个像素点的像素值，得到该第二样本图像的灰度曲线，从而可以从该灰度曲线中获知该第二样本图像的多个像素点的像素值的均值、最大值或最小值等，从而确定异常的像素值，当然，也可 以通过直方图的形式对第二样本图像的多个像素值进行统计，从而确定异常的像素值，例如，可以将与平均值的差值大于预设差值的像素值对应的像素点作为异常像素点，也可以按照与该平均值的差值，将该第二样本图像的多个像素值的80％作为正常值，将20％作为异常值，其中，异常值为与平均值的差值较大的像素值，正常值为与平均值的差值较小的像素值。需要说明的是，上述仅为几种示例性说明，具体地，该步骤一可以通过任一种异常值检测算法实现，也可以通过任一种异常值处理方法实现，本申请实施例对此不作限定。
步骤二、当根据该属性信息确定删除异常像素点后的第二样本图像的灰度范围大于目标范围时，第二初始模型对该第二样本图像进行规范化处理，将该第二样本图像的灰度范围调整为目标范围内。
第二初始模型中还可以设置有目标范围，该目标范围可以由相关技术人员预先设置，例如，该目标范围可以为[0,255]。在第二样本图像的灰度范围大于该目标范围时，则可以对其进行规范化处理，将第二样本图像的灰度范围调整在目标范围内，这样在后续分割过程中所有的第二样本图像的灰度范围均在目标范围内，灰度范围统一，第二样本图像之间也具有可比性，对第二样本图像进行分割得到的分割结果也更准确。具体地，该规范化处理过程可以通过任一种规范化方法实现，例如，可以根据线性函数将灰度范围转换为目标范围内，本申请实施例对此不作限定，也不做过多赘述。
步骤三、当根据该属性信息确定该第二样本图像的通道数量大于一时，第二初始模型将该第二样本图像的每个像素值均减去目标图像均值。
第二样本图像可能为彩色图像，也可以为灰度图像，其中，彩色图像的通道数量大于一，而灰度图像的通道数量为一。如果根据属性信息确定第二样本图像为彩色图像时，还可以将该第二样本图像的每个像素值均减去目标图像均值。在一些实施例中，该目标图像均值可以在对第二初始模型进行预训练的过程中得到，也就是，该目标图像均值可以在上述步骤200中得到。具体地，该目标图像均值可以为预训练时的多个第一样本图像的图像均值，也就是像素值的平均值。
通过该步骤三，可以使得该第二样本图像的像素值的范围(灰度范围)与模型预训练时的第一样本图像的像素值的范围(灰度范围)一致，这样模型训练和模型使用时对图像进行上述处理，均可以使得处理后的图像的灰度范围一致，从而图像分割结果更准确。
步骤四、当根据该属性信息确定该第二样本图像的模态数量大于一时，第二初始模型将该第二样本图像输入模态融合模块，由该模态融合模块从该第二样本图像的多个像素值进行筛选，得到预处理后的第二样本图像的目标数量的像素值，该预处理后的第二样本图像的模态数量为一。
该第二样本图像的模态数量还可能不为一，例如，该第二样本图像通过多种成像原理或多种成像设备得到，例如，电子计算机断层扫描(Computed Tomography，CT)、磁共振成像(Magnetic Resonance Imaging，MRI)、正电子发射型计算机断层显像(Positron Emission Computed Tomography，PET)等。第二初始模型还可以将多模态的图像进行模态融合，从而对融合后的图像进行分割。其中，该模态融合模块为该第二初始模型中的一个模块，当第二样本图像的模态数量大于一时，即可通过该模态融合模块对第二样本图像进行模态融合。
具体地，该步骤四中模态融合模块对第二样本图像的处理过程也可以理解为：模态融合模块可以根据模块参数，从第二样本图像的多个像素值中选择目标数量的像素值作为第二样本图像的像素值。该目标数量为模态数量为一的第二样本图像的像素值的数量。在一些实施例中，该模态融合模块的模块参数可以在模型训练过程中进行调整，使得选择的目标数量的像素值更具有代表性，更能表征该第二样本图像的特征。上述仅提供了一种模态融合方式，具体地，该过程还可以采用其他方式，本申请实施例对此不作限定。
例如，如图3所示，对于模态融合模块，第二样本图像的模态数量可以为n，n为正整数，模态融合模块可以将该n个模态的像素值进行融合，并进行上述选择步骤，最终得到要输入第一图像分割模块和第二分割模块的第二样本图像的像素值，从而将n个模态的第二样本图像融合为一个模态的第二样本图像。其中，modality为模态，H为高度Height，W为宽度Width，D为深度Depth，C为通道数量Channel，Concat为堆叠函数，Input为输入，Convolution为卷积，Featuremap为特征图。
经过上述一个或多个步骤对第二样本图像进行预处理后，可以将预处理后的第二样本图像输入第一图像分割模块和第二图像分割模块中，进行后续分割步骤。当然，该预处理过程也不限定于上述四个步骤，该预处理过程还可以包括其他步骤，例如，第二初始模型可以对第二样本图像进行采样，且可以根据该第二样本图像的尺寸范围确定对该第二样本图像的采样方式，例如，以第二初始模型的降采样倍数被设置为8为例，如图4所示，如果图像调整大小(Resize) 之后降采样8倍的尺寸大于一个像素，则可直接对图像进行Resize。如图5所示，而如果对图像进行Resize之后降采样8倍后尺寸小于一个像素，则说明在该降采样过程中很多有用的信息丢失了，需要采用多尺度的图像剪裁(Crop)方式对图像进行采样。
205、该第一图像分割模块和第二图像分割模块分别对每个第二样本图像进行分割，得到第一分割结果和第二分割结果。
对于第一图像分割模块，该第一图像分割模块可以采用2个阶段(2-stage)流式设计，也就是该第一图像分割模块可以对第二样本图像进行两个阶段的分割，也就是两次分割，在第一阶段第一图像分割模块可以对第二样本图像进行粗分割，在第二阶段第二图像分割模块可以对第二样本图像进行细分割，这样可以应对不同难度的分割任务。
在一些实施例中，该第一图像分割模块可以采用18层的3D Unity Networking(unet)模型实现，图6是本申请实施例提供的一种3D模型的结构示意图，如图6所示，该3D模型可以先对第二样本图像进行特征提取，并基于提取的特征，进行上采样。该特征提取过程可以通过卷积和池化等步骤实现，在该特征提取过程中第二样本图像的尺寸逐渐变小，在该上采样过程中，可以综合之前特征提取过程中第二样本图像的一些数据和提取特征后得到的数据进行上采样，也就是上述上采样过程采用跳跃式传递(skip connection)的方式实现，最终实现对第二样本图像的分割过程。其中，[132,132,116]和[64,64,56]等表示图像的尺寸，包括宽度、高度和深度。而32、64、128、512、258+512等为上一层网络的卷积核的数量，在此不一一说明，本申请实施例在此仅以该3D模型为例，具体该第一图像分割模块还可以采用其他模型实现，本申请实施例对此不作限定，也不做过多赘述。
该第一图像分割模块对该第二样本图像的分割过程可以为：第一图像分割模块基于该第一图像分割模块的模块参数，对该第二样本图像进行两次分类，得到第一分割结果，该两次分类中第一次分类的分类对象为该第二样本图像的所有像素点，第二次分类的分类对象为该第一次分类结果中的前景像素点。这样通过两次分割，先进行粗分割，再进行细分割，从而综合两次分割结果，得到第一分割结果，可以有效提高图像分割的准确性。
其中，该第一图像分割模块可以支持多类型分割，第一图像分割模块可以对第二样本图像的每个像素点进行分类，确定每个像素点为哪种类型，也就是 至少两个类型中的一个类型。该至少两个类型可以分为两类，一类为背景，另一类为前景，相应地，对应的像素点分别为背景像素点和前景像素点，也就是，类型为背景的像素点为背景像素点，类型为前景的像素点为前景像素点。该背景为至少两个类型中的一个类型，前景为该至少两个类型中的背景之外的其它一种或多种类型。
具体地，上述两次分割过程可以包括下述步骤一至步骤三：
步骤一、第一图像分割模块基于该第一图像分割模块的模块参数，对该第二样本图像的每个像素点进行分类，得到第三分割结果，该第三分割结果用于指示该第二样本图像的每个像素点为至少两个类型中每个类型的概率。
该步骤一即为对第二样本图像的粗分割过程，也就是2-stage流式设计中的第一阶段。第一图像分割模块则可以区分该第二样本图像中的哪些像素点为前景，哪些为背景，从而可以确定该第二样本图像中目标区域的外轮廓，从而可以再通过下述步骤二，对步骤一确定的外轮廓内的像素点进一步进行分类，从而更细致地区分外轮廓中的各个像素点的类型，以确定目标区域的具体分布。
步骤二、第一图像分割模块基于该第三分割结果和该第一图像分割模块的模块参数，对该第三分割结果中的每个前景像素点进行分类，得到第四分割结果，该第四分割结果用于指示该第三分割结果中的每个前景像素点为该至少两个类型中每个类型的概率。
在该步骤二中，忽略第三分割结果中的背景像素点，仅对前景像素点再次进行分类，从而更细致地对上述目标区域的外轮廓内的像素点进行分类，得到更细致的分割结果。该步骤二是对第二样本图像的细分割过程，也就是2-stage流式设计中的第二阶段。第一图像分割模块可以对上述被确定为前景的像素点再次进行分类，再次确定每个像素点是上述至少两个类型中的哪一个类型。
步骤三、第一图像分割模块基于该第三分割结果和该第四分割结果，得到第一分割结果。
在经过粗分割和细分割，得到第三分割结果和第四分割结果后，第一图像分割模块还可以综合两次的分割结果，确定该第二样本图像的第一分割结果。在一些实施例中，对于在粗分割中的背景像素点，可以将该第三分割结果中该背景像素点的分割结果作为第一分割结果中该像素点的分割结果。而对于在粗分割中的前景像素点，在第三分割结果和第四分割结果中均存在对这部分像素点的分割结果，则第一图像分割模块可以将这部分像素点在第三分割结果和第 四分割结果中的分割结果取平均值作为第一分割结果。
在一些实施例中，第一图像分割模块也可以直接将第三分割结果和第四分割结果的平均值作为第一分割结果，其中，该第四分割结果中还包括上述第三分割结果中的背景像素点的分割结果，这部分像素点直接确定为背景。也就是，在第二阶段第一图像分割模块并未对第一阶段中的背景像素点再次进行分类，而是直接默认这部分背景像素点的类型为背景。
当然，上述仅以该步骤三通过取平均值的方式实现为例进行说明，在一些实施例中，第一图像分割模块还可以将第三分割结果和第四分割结果进行加权求和，得到第一分割结果，本申请实施例对该步骤三的具体实施方式不作限定。
对于第二图像分割模块，在一些实施例中，该第二图像分割模块可以采用深度残差网络(Deep residual network,ResNet)实现，例如，该ResNet可以为ResNet-18、ResNet-101或ResNet-152，本申请实施例对具体采用哪种ResNet不作限定。
在一些实施例中，该第二图像分割模块可以包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。例如，该第二图像分割模块可以包括两个图像分割子模块：ResNet-18和ResNet-101，其中，ResNet-101的深度大于ResNet-18的深度。又在上述步骤203中已经说明，该图像信息还可以包括该多个第二样本图像的图像数量。这样在第二样本图像的图像数量不同时，可以采用不同深度的图像分割子模块进行训练，从而可以避免出现过拟合(over-fitting)现象或训练后的模型的分割能力差的问题。
具体地，该第二初始模型中还可以存储有图像数量与图像分割子模块的对应关系，相应地，该步骤205中还包括：第二初始模型基于该多个第二样本图像的图像数量，获取该图像数量对应的图像分割子模块作为该第二图像分割模块，该多个第二样本图像的图像数量即为上述步骤203中获取到的图像信息中的一种信息。
在一些实施例中，在该第二图像分割模块的获取步骤中，图像数量越大，获取的图像分割子模块的深度越大。这样可以有效应对小数据的情况，在样本数量很少时也可以训练模型，得到分割效果较好的图像分割模型。
进一步地，以该第二图像分割模块包括两个图像子模块为例，该第二图像分割模块的获取步骤可以为：当该多个第二样本图像的图像数量大于预设数量时，第二初始模型获取第一图像分割子模块；当该多个第二样本图像的图像数 量小于或等于预设数量时，第二初始模型获取第二图像分割子模块。其中，该第一图像分割子模块的深度大于第二图像分割子模块的深度。预设数量可以由相关技术人员预先设置，本申请实施例对该目标数量的具体取值不作限定。
例如，第一图像分割子模块可以为ResNet-101，第二图像分割子模块可以为ResNet-18，以该目标数量为100为例，上述第二图像分割模块的获取步骤可以为：当第二样本图像的图像数量小于100时，可以采用ResNet-18作为基础模型，当第二样本图像的图像数量大于100时，可以采用ResNet-101作为基础模型。该ResNet-18和ResNet-101的结构表分别为下述表一和表二：
表一
Figure PCTCN2019110541-appb-000001
表二
Figure PCTCN2019110541-appb-000002
Figure PCTCN2019110541-appb-000003
其中，Layer name为层的名称，Conv为convolution的缩写，为卷积的意思，stride为步长，blocks为块，max pool为最大池化。其中，以ResNet-18的结构为例进行简单说明，Conv1可以为一个卷积层，卷积核的大小为7x7,卷积核的数量为64,步长为2。Conv2_x的第一层为池化层，池化层后包括两个卷积层，均为64个3x3的卷积核，该两个卷积核即为一个块，该Conv2_x的池化层后包括两个块，也就是该Conv2_x包括一个池化层，四个卷积层。需要说明的是，上述表一和表二中仅示出了Conv1至Conv5_x的结构，在Conv5_x后其实还有一个全卷积(Fully Convolution)FC层并未在表一和表二中示出，在此不多做赘述。
在该ResNet-18和ResNet-101中，Conv3_x的第一层，也就是Conv3_1的步长设置为2，Conv4_x的第一层的步长设置为1，空洞(dilation)设置为2，这样可避免降采样对分割结果的影响，也可以保留ResNet-101各个层的感受野。当然，上述各个卷积层之后都有线性整流函数(Rectified Linear Unit，ReLU)层和批量归一化(Batch Normalization)层，本申请实施例在此不多做赘述。
需要说明的是，上述ResNet-18和ResNet-101的Conv1至Conv5_x均为基础模型，也就是该第二图像分割模块的骨干模型，在Conv5_3之后，还可以对第二样本图像进行降采样，具体地，降采样过程还可以采用多尺度卷积核，例如，可以采用1、9、19、37和74五种倍数的卷积核。通常地，该降采样过程通常通过池化(pool)层实现，在本申请实施例中，可以将所有的pool层均更换为深度卷积(depthwise convolution)层实现。当然，上述设置还可以由相关技术人员根据图像分割需求进行设置或调整，本申请实施例对此不作具体限定。
与步骤200中的内容同理，第二初始模型的模型参数可以基于多个第一样本图像预训练得到，也就是上述Conv1至Conv5_x的参数可以基于多个第一样 本图像预训练得到，在预训练过程中主要对该Conv1至Conv5_x的参数进行训练，对于后面的其它层的参数，可以采用方差为0.01，均值为0的高斯分布数值作为初始值，当然，在此仅提供了一种示例，该其他层的初始值也可以为其他数值，本申请实施例对初始值的设置不作具体限定。
该第二图像分割模块对该第二样本图像的分割过程可以包括下述步骤一和二：
步骤一、第二图像分割模块基于该第二图像分割模块的模块参数，对该第二样本图像进行特征提取。
第二图像分割模块可以基于上述获取到的第二图像分割模块的模块参数，对第二样本图像进行特征提取，得到该第二样本图像的特征，例如，该特征可以为特征图(feature map)的形式。
步骤二、第二图像分割模块基于提取的特征，对该第二样本图像的每个像素点进行分类，得到第二分割结果。
第二图像分割模块提取特征后，还可以进行上述降采样的过程，并在所有信息均组合后，对该第二样本图像的每个像素点进行分类，以确定第二分割结果。
在一些实施例中，该第二图像分割模块用于对2D图像进行分割，如果第二样本图像为3D图像，在该第二图像分割模块对该第二样本图像的分割过程之前，第二初始模型还需要对第二样本图像进行处理，将3D图像处理为2D图像，从而将2D图像输入该第二图像分割模块中。
具体地，当根据该第一图像的属性信息确定该第一图像为三维图像，且确定需要由该第二图像分割模块对该第一图像进行分割时，第二初始模型对该第一图像进行处理，得到多个第一子图像，该第一子图像为二维图像。需要说明的是，该将3D图像处理为多个2D图像的过程可以采用任一种3D/2D转换的方式，例如，可以在某个方向上对3D图像进行采样，得到多个2D图像，当然，也可以在各个方向上均对3D图像进行采样，得到多个2D图像，本申请实施例对此不作限定。
相应地，该第二图像分割模块对第一图像的分割过程包括：第二图像分割模块基于该第二图像分割模块的模块参数，分别对该第一图像对应的多个第一子图像进行分割，得到多个第二子分割结果；第二图像分割模块对该多个子分割结果进行融合，得到第二分割结果。
上述步骤202至步骤205为将该多个第二样本图像输入该第二初始模型中，由该第二初始模型获取该多个第二样本图像的图像信息，根据该图像信息、该第二初始模型中的第一图像分割模块和第二图像分割模块，对每个第二样本图像进行分割的过程，该第二初始模型既包括用于对3D图像进行分割的模块，也包括用于对2D图像进行分割的模块，从而提高了该第二初始模型的适用性和通用性，基于该第二初始模型训练得到的图像分割模型的适用性和通用性也更高，且提供了多种灵活可变的分割方式，提高了图像分割的准确性。
206、计算机设备中的第二初始模型基于该多个第二样本图像的标签、该第一分割结果和该第二分割结果，分别获取第一分割误差和第二分割误差。
在得到第一分割结果和第二分割结果后，第二初始模型可以基于第二样本图像的标签来分别确定该第一分割结果和第二分割结果是否准确，具体地，分割结果是否准确可以通过分割误差来衡量。其中，该第一分割误差为该第一图像分割模块对应的第一分割结果的分割误差，该第二分割误差为该第二图像分割模块对应的第二分割结果的分割误差。
在一些实施例中，该第一分割结果的分割误差的获取过程采用第一损失函数实现，该第一分割误差的获取过程采用第一损失函数实现，该第一损失函数中像素点的每个类型的权重基于该多个第二样本图像的图像信息中的该类型的像素点在该多个第二样本图像中所占的比例确定。例如，该权重可以通过下述公式确定：
Figure PCTCN2019110541-appb-000004
其中，w c为类型c的权重，N为第二样本图像的图像数量，i为第二样本图像的标识，t c，i为第二样本图像i中类型c的像素点数量，n i为第二样本图像i中的所有像素点的数量，∑为累加函数或求和函数。
在一些实施例中，该第二分割结果的分割误差的获取过程采用第二损失函数实现，该第二损失函数的权重基于在线难样本挖掘(Online Hard Example Mining，OHEM)算法确定，这样可以有效对第二样本图像中的难样本进行区别，并降低这部分样本对模型的参数的影响，从而可以应对样本标签不均衡带来的不良影响。
例如，该第二损失函数可以为交叉熵函数(cross entropy function)，上述第一损失函数也可以为cross entropy function，也可以为其他损失函数。在一些实施例中，第一损失函数和第二损失函数可以相同，也可以不同，本申请实施 例对第一损失函数和第二损失函数具体采用哪个损失函数，以及第一损失函数和第二损失函数是否相同不作限定。
207、计算机设备中的第二初始模型分别基于该第一分割误差和该第二分割误差，对该第一图像分割模块和第二图像分割模块的模块参数进行调整，直至达到第一迭代停止次数时停止，得到第一图像分割模块和第二图像分割模块。
第二初始模型在获知第一分割结果和第二分割结果是否准确后，可以对两个图像分割模块的模块参数进行调整，以使得多次调整后的模块参数，可以使得第一图像分割模块和第二图像分割模块对第二样本图像的分割结果更准确。
在一些实施例中，该第一迭代停止次数基于交叉验证的方式确定。具体地，该第一迭代停止次数可以基于k-折交叉验证的方式确定，例如，可以基于五折交叉验证的方式确定。以五折交叉验证为例，可以将第二样本图像分为五部分，将其中四部分作为训练集，将另外一部分作为验证集，再以另外的组合方式进行多次训练和验证，当然，也可以确定不同组合方式后，同时以不同的组合方式对第二初始模型进行训练和验证，这样通过对样本数据的多种组合进行训练和验证，使得该第二初始模型遍历了所有的样本数据，训练后的模型的通用性更好，分割结果更准确。其中，该交叉验证过程主要为每进行一定次数的迭代过程时，通过验证数据对训练的模型进行验证，如果分割误差符合目标条件，则可以停止，如果不符合，则可以继续进行上述迭代过程，本申请实施例在此不作过多赘述。
上述步骤203至步骤207是基于该多个第二样本图像对该第二初始模型中的该第一图像分割模块和该第二图像分割模块进行训练，达到第一迭代停止次数时停止，得到第一图像分割模块和第二图像分割模块的模块参数的过程，在该过程中，该第一图像分割模块的模块参数基于每次迭代过程中的第一分割误差进行调整得到，该第二图像分割模块的模块参数基于次迭代过程中的第二分割误差进行调整得到。第二初始模型每执行一遍该步骤203至步骤207即为一次迭代过程，第二初始模型可以多次执行上述过程，通过多次迭代，对两个图像分割模块的模块参数进行调整，也就是，分别训练第一图像分割模块和第二图像分割模块的过程。
在一些实施例中，上述过程中，计算机设备在对两个图像分割模块的模块参数进行调整时，还可以对模态融合模块的模块参数进行调整，从而在这个训练过程中训练得到模态融合模块的模块参数。
在具体的示例中，上述第一图像分割模块和第二图像分割模块均可以为卷积神经网络模型，上述每次迭代过程中，模型均可以计算预测的结果的误差，并反向传播至卷积神经网络模型中，从而可以通过梯度下降算法求解神经网络模型的卷积模板参数w和偏置参数b。
208、计算机设备中的第二初始模型基于训练得到的第一图像分割模块、第二图像分割模块对该多个第二样本图像进行分割，得到每个第二样本图像的第一分割结果和第二分割结果。
由于该第一图像分割模块和第二图像分割模块分别适用于对3D图像和2D图像进行分割，则可能对于一个第二样本图像，第一图像分割模块对该第二样本图像的分割结果较为准确，而第二图像分割模块对该第二样本图像的分割结果很不准确，这样如果第二初始模型直接采用两个模块的综合结果，可能得到的最终的分割结果受到第二图像分割模块的分割结果的影响，导致最终分割结果的准确性降低。
在第一图像分割模块和第二图像分割模块训练完成后，第二初始模型还可以基于训练完成的两个模块，训练两个模块的混合策略，也就是训练对于一个第二样本图像，选择哪个模块或两个模块来对该第二样本图像进行分割更佳。
第二初始模型可以使用训练完成的两个模块分别对第二样本图像进行分割，得到第一分割结果和第二分割结果，并对该两个分割结果以及两个分割结果的综合分割结果进行评估，判断哪种模块选择方式得到的分割结果更准确。
209、计算机设备中的第二初始模型基于该第一分割结果和该第二分割结果，得到第五分割结果。
其中，该第五分割结果为该第一分割结果和第二分割结果的综合分割结果。在一些实施例中，第二初始模型获取第五分割结果的过程可以为：第二初始模型将第一分割结果和第二分割结果的平均值作为第五分割结果，也就是对于每个像素点为每个类型的概率，可以将第一分割结果中的概率和第二分割结果中的概率的平均值作为第五分割结果中的概率。
在一些实施例中，该第一分割结果和第二分割结果还可以对应有权重，第二初始模型获取第五分割结果的过程可以为：第二初始模型对第一分割结果和第二分割结果进行加权求和，得到第五分割结果。
当然，上述仅提供了两种示例，该第五分割结果的获取过程还可以通过其他方式实现，本申请实施例对此不作限定。
210、计算机设备中的第二初始模型基于第二样本图像的标签、每个第二样本图像的第一分割结果、第二分割结果和该第五分割结果，获取第一分割误差、第二分割误差和第三分割误差。
其中，该第三分割误差为该第五分割结果的分割误差。第二初始模型在得到第一分割结果、第二分割结果和第五分割结果后，则可以基于第二样本图像的标签，分别确定各个分割结果的分割误差，以判断各个分割结果是否准确。该各个分割结果的分割误差也可以通过上述第一损失函数或第二损失函数获取得到，本申请实施例在此不作过多赘述。
211、计算机设备中的第二初始模型基于该第一分割误差、该第二分割结果的分割误差和该第三分割误差，对该第二初始模型中的模块选择参数进行调整，直至达到第二迭代停止次数时停止，得到图像分割模型。
其中，该模块选择参数用于决策选择该第一图像分割模块和该第二图像分割模块中至少一个分割模块对第一图像进行分割。这样第二初始模型基于各个分割结果的分割误差，对模块选择参数进行调整，在多次调整后，得到的图像分割模型即可自行决策如何对模块进行选择可以使得对第二样本图像的分割结果更准确。
上述步骤208至步骤211是基于该多个第二样本图像和训练得到的该第一图像分割模块和该第二图像分割模块，对该第二初始模型中的模块选择参数进行训练，直到达到第二迭代停止次数时停止，得到图像分割模型，该模块选择参数用于决策选择该第一图像分割模块和该第二图像分割模块中至少一个分割模块对第一图像进行分割的过程，该过程为对模块选择参数进行训练的过程，该模块选择参数基于训练后的第一图像分割模块、第二图像分割模块和该多个第二样本图像训练得到。
在一些实施例中，该第二迭代停止次数也可以基于交叉验证的方式确定。具体地，该第二迭代停止次数也可以基于k-折交叉验证的方式确定，例如，可以基于五折交叉验证的方式确定，本申请实施例在此不作过多赘述。
综上，该第二初始模型的模型参数包括该第一图像分割模块的模块参数、该第二图像分割模块的模块参数、该第二初始模型中的模态融合模块的模块参数和模块选择参数。则上述步骤202至步骤211也就是基于多个第二样本图像，对该第二初始模型进行训练，得到图像分割模型的过程。
在一些实施例中，上述第二初始模型对第二样本图像分割后还可以进一步 对分割结果进行后处理，从而得到最终的分割图像。也就是，第二初始模型可以基于第一分割结果和第二分割结果中至少一个分割结果得到第二样本图像对应的第三图像，从而基于第二样本图像对应的第三图像，确定最终输出第二样本图像对应的第二图像，该第二图像就是第二样本图像对应的分割后的图像。具体地，该第二样本图像对应的第三图像可以为第一分割结果对应的图像，也可以为第二分割结果对应的图像，还可以是对第一分割结果和第二分割结果进行取平均或加权求和后得到的图像。该后处理过程可以基于上述步骤203中获取到的图像信息中的目标区域的分布信息进行。
也就是，第二初始模型可以基于该第二样本图像对应的第三图像中的多个目标区域和该图像信息所指示的该多个目标区域的分布信息，对该第二样本图像对应的第三图像进行后处理，得到第二样本图像对应的第二图像，该目标区域为该第二样本图像对应的第三图像中目标类型的像素点所在区域，该第二样本图像对应的第二图像中多个目标区域的分布类型、目标区域的数量和目标区域的大小范围与该多个目标区域的分布信息相同。这样根据先验知识，在该后处理过程中可以对分割过程中分类错误的像素点进行进一步修正，使得分割结果更准确。
具体地，该后处理过程可以包括下述任一个或多个步骤：当该第二样本图像对应的第三图像中目标区域的数量或大小范围与该图像信息所指示的该多个目标区域的数量或大小范围不同时，第二初始模型将该第二样本图像对应的第三图像中不符合该多个目标区域的数量或大小范围不同的部分滤除；或，当任一目标区域内部存在背景像素点时，第二初始模型将该背景像素点更改为该目标区域对应的目标类型的像素点。
例如，如图7所示，如果根据目标区域的分布信息确定第一目标区域和第二目标区域的分布类型为完全嵌套型，也就是第二目标区域应在第一目标区域的内部，如果第二样本图像对应的第三图像中有在第一目标区域外的第二目标区域，则可以滤除该在第一目标区域外的第二目标区域。如图8所示，如果根据目标区域的分布信息确定第一目标区域和第二目标区域的分布类型为完全分离型，也就是第二目标区域应在第一目标区域的外部，如果第二样本图像对应的第三图像中有在第一目标区域内部的第二目标区域，则可以将该在第一目标区域内部的第二目标区域填充为第一目标区域。当然，在目标区域内的像素点应该为前景像素点，如果第二样本图像对应的第三图像中目标区域内存在背景 像素点，则可以对该背景像素点进行更正。例如，以人体组织为例，人体组织部位不应该存在空洞，如果第二样本图像对应的第三图像中的人体组织部位存在空洞，则可以对该部分进行填充，以修正分割结果。当然，也可以包括其他步骤，本申请实施例在此不一一列举。
至此，图像分割模型训练完成，训练过程中图像分割模型获取得到该多个第二样本图像的图像信息，该步骤211后，当获取到待分割的第一图像时，该计算机设备可以调用该图像分割模型，由该图像分割模型根据该图像信息，对该第一图像进行分割，输出第二图像，具体地，该图像分割模型具体如何对第一图像进行分割，可以参见下述图10所示实施例，且分割过程与该图像分割模型训练过程中的一些步骤同理，本申请实施例在此不作过多赘述。
图9是本申请实施例提供的图像分割模型的架构示意图，参见图9，该图像分割模型中包括3D网络(Net)和2D网络(Net)，也就是第一图像分割模块和第二图像分割模块，其中，3D Net可以采用2-stage流式设计，经过粗预测后再进行细预测，也就是进行粗分割后再进行细分割。对于输入的样本数据，可以将样本数据输入3D Net和2D Net，经过两个网络后分别得到概率图后，可以采用不同的混合策略融合概率图，也就是，可以训练模块选择参数，确定选择单一网络还是选择两个网络。当然，在两个网络进行分割之前，图像分割模型可以先对样本数据进行预处理，在两个网络进行分割后，图像分割模型还可以对结果进行后处理，从而得到最终输出的分割图像。
本申请实施例提供的图像分割模型的通用性强，在应用于医学图像分割时对医学图像具有场景针对性，也就是对人体组织图像具有场景针对性，且模型可自动训练,用户只需要提供数据即可自动训练，不需要人工参与参数调整。且本申请实施例提供的图像分割模型已经在10种不同的医疗场景中得到了验证，均具有较好的分割效果。且该图像分割模型可以自动化扩展到其他医疗应用场景中，具有很强的医疗影像普适性。
本申请实施例通过以多种人体组织图像对初始模型进行预训练，使得初始模型具备了关于人体组织的先验知识，在需要对某种人体组织图像进行分割时，直接基于这种人体组织图像对预训练后的模型进行训练即可，而无需人工对这种人体组织图像进行分析，再基于分析结果重新设计模型，有效提高了图像分割模型的通用性、适用性和实用性。进一步地，该图像分割模型中包括第一图像分割模块和第二图像分割模块，对于三维图像和二维图像均可以进行准确分 割，进一步地提高了图像分割模型的通用性、适用性和实用性，也提高了图像分割模型的分割准确性。
上述所有可选技术方案，可以采用任意结合形成本申请的可选实施例，在此不再一一赘述。
图10是本申请实施例提供的一种图像分割方法的流程图，该图像分割方法应用于计算机设备中，该计算机设备可以为上述实施环境中的计算机设备101。在本申请实施例中，主要对当获取到待分割的第一图像时，调用该图像分割模型，由该图像分割模型根据该图像信息，对该第一图像进行分割，输出第二图像的过程进行了详细说明，在本申请实施例中仅以该图像分割模型包括第一图像分割模块和第二图像分割模块为例进行说明。参见图10，该图像分割方法可以包括以下步骤：
1001、计算机设备获取待分割的第一图像。
计算机设备在检测到图像分割操作时执行该步骤1001，也可以接收用户导入的待分割的第一图像，还可以接收其他计算机设备发送的图像分割请求，该图像分割请求中携带有待分割的第一图像，从该图像分割请求中提取待分割的第一图像，或该图像分割请求中可以携带有该第一图像的相关信息，计算机设备可以基于该相关信息，执行该步骤1001，当然，该计算机设备也可以通过成像原理获取得到待分割的第一图像。本申请实施例对该待分割的第一图像的具体获取方式和获取时机不作限定。
例如，其他计算机设备可以通过成像原理获取得到待分割的第一图像，并向该计算机设备发送该待分割的第一图像，该计算机设备获取到该待分割的第一图像，该第一图像可以为上述目标人体组织图像，这样可以执行下述步骤，利用通过该目标人体组织的样本图像进行训练得到的图像分割模型，对该第一图像进行分割。
1002、计算机设备调用图像分割模型。
其中，该图像分割模型包括第一图像分割模块和第二图像分割模块。该第一图像分割模块和第二图像分割模块分别对应于一种分割算法，该第一图像分割模块用于对三维图像进行分割，该第二图像分割模块用于对二维图像进行分割。
该计算机设备中可以预先存储有图像分割模型，在一些实施例中，该计算 机设备是图2所示的计算机设备，也就是该计算机设备上存储的图像分割模型是在该计算机设备上训练得到的。在一些实施例中，该计算机设备不是图2所示的计算机设备，也即是可以在其他计算机设备上训练得到图像分割模型，该计算机设备可以从其他计算机设备上获取该训练好的图像分割模型。当然，该计算机设备上也可以没有存储有图像分割模型，在该计算机设备获取到待分割的第一图像，需要对第一图像进行分割时，可以实时从其他计算机设备处调用图像分割模型，本申请实施例对此不作限定。
1003、计算机设备将该第一图像输入该图像分割模型中，由该图像分割模型获取该第一图像的属性信息。
与上述步骤203中的内容同理，图像分割模型可以获取第一图像的属性信息，不同的是，该图像分割模型为训练完成的模型，而不是第二初始模型，且该图像分割模型使用时，可以获取第一图像的属性信息，而无需获取图像数量、目标区域的分布信息等。同理地，该属性信息可以包括灰度范围、模态数量和尺寸范围等，本申请实施例对此不作限定。
1004、计算机设备中的图像分割模型根据该第一图像的属性信息，对该第一图像进行预处理。
与步骤204中的内容同理，该图像分割模型对第一图像的预处理过程也可以包括以下任一个或多个步骤：
步骤一、当根据该属性信息确定该第一图像中存在异常像素点时，图像分割模型将该异常像素点删除.
步骤二、当根据该属性信息确定删除异常像素点后的第一图像的灰度范围大于目标范围时，图像分割模型对该第一图像进行规范化处理，将该第一图像的灰度范围调整为目标范围内。
步骤三、当根据该属性信息确定该第一图像的通道数量大于一时，图像分割模型将该第一图像的每个像素值均减去目标图像均值。
步骤四、当根据该属性信息确定该第一图像的模态数量大于一时，图像分割模型将该第一图像输入模态融合模块，由该模态融合模块从该第一图像的多个像素值进行筛选，得到预处理后的第一图像的目标数量的像素值，该预处理后的第一图像的模态数量为一。
该步骤1004中的步骤一至步骤四均与上述步骤204中的步骤一至步骤四同理，本申请实施例在此不作过多赘述。
1005、计算机设备中的图像分割模型将预处理后的第一图像输入该第一图像分割模块和第二图像分割模块中至少一个模块中，由该第一图像分割模块和第二图像分割模块中至少一个模块对该第一图像进行分割，得到第三图像。
由于适用于该第一图像进行分割的模块可能只有第一图像分割模块，也可能只有第二图像分割模块，也可以是两个模块，则图像分割模型可以基于模块选择参数选择第一图像分割模块和第二图像分割模块中至少一个模块对该第一图像进行分割。具体地，该步骤1005可能包括以下三种可能情况：
第一种情况、图像分割模型基于该图像分割模型的模块选择参数，由该第一图像分割模块对该第一图像进行分割，得到第一分割结果，基于该第一分割结果，得到第三图像，该第一分割结果用于指示该第一图像的每个像素点为至少两个类型中每个类型的概率。
第二种情况、图像分割模型基于该图像分割模型的模块选择参数，由该第二图像分割模块对该第一图像进行分割，得到第二分割结果，基于该第二分割结果，得到第三图像，该第二分割结果用于指示该第一图像的每个像素点为至少两个类型中每个类型的概率。
第三种情况、图像分割模型基于该图像分割模型的模块选择参数，分别由该第一图像分割模块和第二图像分割模块对该第一图像进行分割，得到第一分割结果和第二分割结果，基于该第一分割结果和第二分割结果，得到第三图像。
当然，基于第一分割结果和第二分割结果，得到第三图像的过程也与上述步骤211中的内容同理，上述三种情况分别对应于该第三图像的三种获取过程，分别为：该第三图像为第一分割结果对应的图像、该第三图像为第二分割结果对应的图像，该第三图像为对第一分割结果和第二分割结果进行取平均或加权求和后得到的图像，本申请实施例在此不作过多赘述。
与上述步骤205中的内容同理，该第一图像分割模块对该第一图像的分割过程可以为：第一图像分割模块基于该第一图像分割模块的模块参数，对该第一图像进行两次分类，得到第一分割结果，该两次分类中第一次分类的分类对象为该第一图像的所有像素点，第二次分类的分类对象为该第一次分类结果中的前景像素点。具体可以包括下述步骤一至步骤三：
步骤一、第一图像分割模块基于该第一图像分割模块的模块参数，对该第一图像的每个像素点进行分类，得到第三分割结果，该第三分割结果用于指示该第一图像的每个像素点为至少两个类型中每个类型的概率，该至少两个类型 包括前景和背景，该前景为背景之外的任一类型。
步骤二、第一图像分割模块基于该第三分割结果和该第一图像分割模块的模块参数，对该第三分割结果中的每个前景像素点进行分类，得到第四分割结果，该第四分割结果用于指示该第三分割结果中的每个前景像素点为该至少两个类型中每个类型的概率。
步骤三、第一图像分割模块基于该第三分割结果和该第四分割结果，得到第一分割结果。
与上述步骤205中的内容同理，该第二图像分割模块对该第一图像的分割过程可以包括下述步骤一和二：
步骤一、第二图像分割模块基于该第二图像分割模块的模块参数，对该第一图像进行特征提取。
步骤二、第二图像分割模块基于提取的特征，对该第一图像的每个像素点进行分类，得到第二分割结果。
同理地，当根据该第一图像的属性信息确定该第一图像为三维图像，且确定需要由该第二图像分割模块对该第一图像进行分割时，图像分割模型可以对该第一图像进行处理，得到多个第一子图像，该第一子图像为二维图像。相应地，该第二图像分割模块对第一图像的分割过程包括：第二图像分割模块基于该第二图像分割模块的模块参数，分别对该第一图像对应的多个第一子图像进行分割，得到多个第二子分割结果；第二图像分割模块对该多个子分割结果进行融合，得到第二分割结果。
1006、计算机设备中的图像分割模型根据该图像分割模型中的多个第二样本图像的图像信息，对该第三图像进行后处理，输出第二图像。
与步骤211中的后处理过程同理，图像分割模型也可以对第三图像进行后处理，同理地，该后处理过程也可以为：图像分割模型基于该第三图像中的多个目标区域和该图像信息所指示的该多个目标区域的分布信息，对该第三图像进行后处理，得到第二图像，该目标区域为该第三图像中目标类型的像素点所在区域，该第二图像中多个目标区域的分布类型、目标区域的数量和目标区域的大小范围与该多个目标区域的分布信息相同。
与步骤211中的后处理过程同理，该步骤1006中，图像分割模型也可以执行下述任一个或多个步骤：当该第三图像中目标区域的数量或大小范围与该图像信息所指示的该多个目标区域的数量或大小范围不同时，图像分割模型将该 第三图像中不符合该多个目标区域的数量或大小范围不同的部分滤除；或，当任一目标区域内部存在背景像素点时，将该背景像素点更改为该目标区域对应的目标类型的像素点。
上述步骤1003至步骤1006是基于该图像分割模型中的第一图像分割模块和第二图像分割模块中至少一个模块，以及该图像信息，对该第一图像进行分割，输出第二图像的过程，在得到第二图像之后，计算机设备可以存储该第二图像，当然，也可以将第一图像和第二图像对应存储，如果该计算机设备为基于其他计算机设备的图像分割请求进行的上述图像分割过程，也可以将该第二图像发送至该其他计算机设备。
需要说明的是，在本申请实施例中仅以该图像分割模型包括第一图像分割模块和第二图像分割模块为例进行说明，该图像分割模型还可以仅包括一个图像分割模块或包括更多个图像分割模块，图像分割流程均与上述过程同理，在此不作过多赘述。
本申请实施例通过以多种人体组织图像对初始模型进行预训练，使得初始模型具备了关于人体组织的先验知识，在需要对某种人体组织图像进行分割时，直接基于这种人体组织图像对预训练后的模型进行训练即可，而无需人工对这种人体组织图像进行分析，再基于分析结果重新设计模型，且上述方法得到的图像分割模型则可以对这种人体组织图像进行准确分割，有效提高了图像分割方法的通用性、适用性和实用性，也有效提高了图像分割方法的准确性。
上述所有可选技术方案，可以采用任意结合形成本申请的可选实施例，在此不再一一赘述。
应该理解的是，本申请各实施例中的各个步骤并不是必然按照步骤标号指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，各实施例中至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，还提供了一种计算机设备，该计算机设备包括图像分割装置，图像分割装置中包括各个模块，每个模块可全部或部分通过软件、硬件或其组合来实现。
图11是本申请实施例提供的一种图像分割装置的结构示意图，参见图11，该装置包括：
训练模块1101，用于基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型，该多个第一样本图像包括多种人体组织图像，该第二初始模型中包括该多种人体组织对应的多个目标区域的分布信息；
该训练模块1101，还用于基于多个第二样本图像，对该第二初始模型进行训练，得到图像分割模型，该多个第二样本图像为目标人体组织图像，训练过程中该图像分割模型获取得到该多个第二样本图像的图像信息，该多个第二样本图像的图像信息至少包括该目标人体组织对应的多个目标区域的分布信息；
分割模块1102，用于当获取到待分割的第一图像时，调用该图像分割模型，由该图像分割模型根据该图像信息，对该第一图像进行分割，输出第二图像。
在一些实施例中，该第一初始模型、该第二初始模型和该图像分割模型均包括第一图像分割模块和第二图像分割模块，该第一图像分割模块和第二图像分割模块分别对应于一种分割算法，该第一图像分割模块用于对三维图像进行分割，该第二图像分割模块用于对二维图像进行分割；
相应地，该分割模块1102用于基于该图像分割模型中的第一图像分割模块和第二图像分割模块中至少一个模块，以及该图像信息，对该第一图像进行分割，输出第二图像。
在一些实施例中，该分割模块1102用于：
根据该第一图像的属性信息，对该第一图像进行预处理；
将预处理后的第一图像输入该第一图像分割模块和第二图像分割模块中至少一个模块中，由该第一图像分割模块和该第二图像分割模块中至少一个模块对该第一图像进行分割，得到第三图像；
基于该图像信息，对该第三图像进行后处理，输出第二图像。
在一些实施例中，该分割模块1102用于：
当根据该属性信息确定该第一图像中存在异常像素点时，将该异常像素点删除；
当根据该属性信息确定删除异常像素点后的第一图像的灰度范围大于目标范围时，对该第一图像进行规范化处理，将该第一图像的灰度范围调整为目标范围内；
当根据该属性信息确定该第一图像的通道数量大于一时，将该第一图像的每个像素值均减去目标图像均值；
当根据该属性信息确定该第一图像的模态数量大于一时，将该第一图像输入模态融合模块，由该模态融合模块从该第一图像的多个像素值进行筛选，得到预处理后的第一图像的目标数量的像素值，该预处理后的第一图像的模态数量为一。
在一些实施例中，该分割模块1102用于基于该第三图像中的多个目标区域和该图像信息所指示的该多个目标区域的分布信息，对该第三图像进行后处理，得到第二图像，该目标区域为该第三图像中目标类型的像素点所在区域，该第二图像中多个目标区域的分布类型、目标区域的数量和目标区域的大小范围与该多个目标区域的分布信息相同。
在一些实施例中，该分割模块1102用于：
当该第三图像中目标区域的数量或大小范围与该图像信息所指示的该多个目标区域的数量或大小范围不同时，将该第三图像中不符合该多个目标区域的数量或大小范围不同的部分滤除；或，
当任一目标区域内部存在背景像素点时，将该背景像素点更改为该目标区域对应的目标类型的像素点。
在一些实施例中，该分割模块1102用于：
基于该图像分割模型的模块选择参数，由该第一图像分割模块，对该第一图像进行分割，得到第一分割结果，基于该第一分割结果，得到第三图像，该第一分割结果用于指示该第一图像的每个像素点为至少两个类型中每个类型的概率；或，
基于该图像分割模型的模块选择参数，由该第二图像分割模块，对该第一图像进行分割，得到第二分割结果，基于该第二分割结果，得到第三图像，该第二分割结果用于指示该第一图像的每个像素点为至少两个类型中每个类型的概率；或，
基于该图像分割模型的模块选择参数，分别由该第一图像分割模块和第二图像分割模块对该第一图像进行分割，得到第一分割结果和第二分割结果，基于该第一分割结果和第二分割结果，得到第三图像。
在一些实施例中，该分割模块1102用于：
基于该第一图像分割模块的模块参数，对该第一图像进行两次分类，得到 第一分割结果，该两次分类中第一次分类的分类对象为该第一图像的所有像素点，第二次分类的分类对象为该第一次分类结果中的前景像素点；
在一些实施例中，该分割模块1102用于：
基于该第二图像分割模块的模块参数，对该第一图像进行特征提取；
基于提取的特征，对该第一图像的每个像素点进行分类，得到第二分割结果。
在一些实施例中，该训练模块1101用于：
基于该多个第二样本图像对该第二初始模型中的该第一图像分割模块、该第二图像分割模块进行训练，直到达到第一迭代停止次数时停止，得到第一图像分割模块和第二图像分割模块的模块参数；
基于该多个第二样本图像和训练得到的该第一图像分割模块和该第二图像分割模块，对该第二初始模型中的模块选择参数进行训练，直到达到第二迭代停止次数时停止，得到图像分割模型，该模块选择参数用于决策选择该第一图像分割模块和该第二图像分割模块中至少一个分割模块对第一图像进行分割。
在一些实施例中，该图像信息还包括每个第二样本图像的属性信息；
相应地，该训练模块1101还用于基于该每个第二样本图像的属性信息，对每个第二样本图像进行预处理，将预处理后的多个第二样本图像输入该第一图像分割模块和该第二图像分割模块中。
在一些实施例中，该第一图像分割模块的模块参数基于每次迭代过程中的第一分割误差进行调整得到，该第一分割误差为该第一图像分割模块对应的第一分割结果的分割误差，该第一分割误差的获取过程采用第一损失函数实现，该第一损失函数中像素点的每个类型的权重基于该多个第二样本图像的图像信息中的该类型的像素点在该多个第二样本图像中所占的比例确定；
该第二图像分割模块的模块参数基于次迭代过程中的第二分割误差进行调整得到，该第二分割误差为该第二图像分割模块对应的第二分割结果的分割误差，该第二分割误差的获取过程采用第二损失函数实现，该第二损失函数的权重基于在线难样本挖掘OHEM算法确定；
该第一迭代停止次数和该第二迭代停止次数基于交叉验证的方式确定。
在一些实施例中，该图像信息还包括该多个第二样本图像的图像数量；
相应地，该训练模块1101还用于基于该图像数量，获取该图像数量对应的图像分割子模块作为该第二图像分割模块进行训练，该第二图像分割模块中包 括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
本申请实施例提供的装置，通过以多种人体组织图像对初始模型进行预训练，使得初始模型具备了关于人体组织的先验知识，在需要对某种人体组织图像进行分割时，直接基于这种人体组织图像对预训练后的模型进行训练即可，而无需人工对这种人体组织图像进行分析，再基于分析结果重新设计模型，且上述方法得到的图像分割模型则可以对这种人体组织图像进行准确分割，有效提高了图像分割方法的通用性、适用性和实用性，也有效提高了图像分割方法的准确性。
需要说明的是：上述实施例提供的图像分割装置在分割图像时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将计算机设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的图像分割装置与图像分割方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
上述计算机设备可以被提供为下述图12所示的终端，也可以被提供为下述图13所示的服务器，本申请实施例对此不作限定。
图12是本申请实施例提供的一种终端的结构示意图。该终端1200可以是：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑或台式电脑。终端1200还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1200包括有：处理器1201和存储器1202。
处理器1201可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器1201可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1201也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施 例中，处理器1201可以集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器1201还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1202可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1202还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1202中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1201所执行以实现本申请中方法实施例提供的图像分割方法或图像分割模型训练方法。
在一些实施例中，终端1200还可选包括有：外围设备接口1203和至少一个外围设备。处理器1201、存储器1202和外围设备接口1203之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口1203相连。具体地，外围设备包括：射频电路1204、触摸显示屏1205、摄像头1206、音频电路1207、定位组件1208和电源1209中的至少一种。
外围设备接口1203可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器1201和存储器1202。在一些实施例中，处理器1201、存储器1202和外围设备接口1203被集成在同一芯片或电路板上；在一些其他实施例中，处理器1201、存储器1202和外围设备接口1203中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路1204用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路1204通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1204将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路1204包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路1204可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1204还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1205用于显示UI(User Interface，用户界面)。该UI可以包括图 形、文本、图标、视频及其它们的任意组合。当显示屏1205是触摸显示屏时，显示屏1205还具有采集在显示屏1205的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1201进行处理。此时，显示屏1205还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏1205可以为一个，设置终端1200的前面板；在另一些实施例中，显示屏1205可以为至少两个，分别设置在终端1200的不同表面或呈折叠设计；在一些实施例中，显示屏1205可以是柔性显示屏，设置在终端1200的弯曲表面上或折叠面上。甚至，显示屏1205还可以设置成非矩形的不规则图形，也即异形屏。显示屏1205可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件1206用于采集图像或视频。可选地，摄像头组件1206包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件1206还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路1207可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器1201进行处理，或者输入至射频电路1204以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端1200的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器1201或射频电路1204的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路1207还可以包括耳机插孔。
定位组件1208用于定位终端1200的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件1208可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯 的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源1209用于为终端1200中的各个组件进行供电。电源1209可以是交流电、直流电、一次性电池或可充电电池。当电源1209包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1200还包括有一个或多个传感器1210。该一个或多个传感器1210包括但不限于：加速度传感器1211、陀螺仪传感器1212、压力传感器1213、指纹传感器1214、光学传感器1215以及接近传感器1216。
加速度传感器1211可以检测以终端1200建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器1211可以用于检测重力加速度在三个坐标轴上的分量。处理器1201可以根据加速度传感器1211采集的重力加速度信号，控制触摸显示屏1205以横向视图或纵向视图进行用户界面的显示。加速度传感器1211还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器1212可以检测终端1200的机体方向及转动角度，陀螺仪传感器1212可以与加速度传感器1211协同采集用户对终端1200的3D动作。处理器1201根据陀螺仪传感器1212采集的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器1213可以设置在终端1200的侧边框和/或触摸显示屏1205的下层。当压力传感器1213设置在终端1200的侧边框时，可以检测用户对终端1200的握持信号，由处理器1201根据压力传感器1213采集的握持信号进行左右手识别或快捷操作。当压力传感器1213设置在触摸显示屏1205的下层时，由处理器1201根据用户对触摸显示屏1205的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器1214用于采集用户的指纹，由处理器1201根据指纹传感器1214采集到的指纹识别用户的身份，或者，由指纹传感器1214根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器1201授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器1214可以被设置终端1200的正面、背面或侧面。当终端1200上设置有物理按键或厂商Logo时，指纹传感器1214可以与物 理按键或厂商Logo集成在一起。
光学传感器1215用于采集环境光强度。在一个实施例中，处理器1201可以根据光学传感器1215采集的环境光强度，控制触摸显示屏1205的显示亮度。具体地，当环境光强度较高时，调高触摸显示屏1205的显示亮度；当环境光强度较低时，调低触摸显示屏1205的显示亮度。在另一个实施例中，处理器1201还可以根据光学传感器1215采集的环境光强度，动态调整摄像头组件1206的拍摄参数。
接近传感器1216，也称距离传感器，通常设置在终端1200的前面板。接近传感器1216用于采集用户与终端1200的正面之间的距离。在一个实施例中，当接近传感器1216检测到用户与终端1200的正面之间的距离逐渐变小时，由处理器1201控制触摸显示屏1205从亮屏状态切换为息屏状态；当接近传感器1216检测到用户与终端1200的正面之间的距离逐渐变大时，由处理器1201控制触摸显示屏1205从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图12中示出的结构并不构成对终端1200的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。图13是本申请实施例提供的一种服务器的结构示意图，该服务器1300可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器(central processing units，CPU)1301和一个或一个以上的存储器1302，其中，该存储器1302中存储有至少一条指令，该至少一条指令由该处理器1301加载并执行以实现上述每个方法实施例提供的图像分割方法或图像分割模型训练方法。当然，该服务器还可以具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该服务器还可以包括其他用于实现设备功能的部件，在此不做赘述。
上述计算机设备可以被提供为下述图14所示的服务器，也可以被提供为下述图15所示的终端，本申请实施例对此不作限定。
上述计算机设备可以被提供为下述图14所示的服务器。如图14所示，该服务器包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的 运行提供环境。该计算机设备的数据库用于存储图像数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像分割方法或图像分割模型训练方法。
上述计算机设备可以被提供为下述图15所示的终端。如图15所示，该终端包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像分割方法或图像分割模型训练方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图14、15示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的服务器、终端的限定，具体的服务器、终端可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，本申请提供的图像分割装置可以实现为一种计算机可读指令的形式，计算机可读指令可在如图14所示的服务器上运行，也可以在如图15所示的终端上运行。服务器或终端的存储器中可存储组成该图像分割装置的各个程序模块，比如训练模块1101和分割模块1102。各个程序模块构成的计算机可读指令使得处理器执行本说明书中描述的本申请各个实施例的图像分割方法或图像分割模型训练方法中的步骤。
本申请实施例提供了一种计算机可读存储介质，所述存储介质中存储有计算机可读指令，该计算机可读指令由处理器加载并具有以实现上述实施例的图像分割方法或图像分割模型训练方法中所具有的操作。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存 储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
本领域技术人员在考虑说明书及实践这里公开的申请后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (30)

  1. 一种图像分割方法，所述方法包括：
    基于多个第一样本图像，计算机设备对第一初始模型进行预训练，得到第二初始模型，所述多个第一样本图像包括多种人体组织图像，所述第二初始模型中包括所述多种人体组织对应的多个目标区域的分布信息；
    基于多个第二样本图像，所述计算机设备对所述第二初始模型进行训练，得到图像分割模型，所述多个第二样本图像为目标人体组织图像，训练过程中所述图像分割模型获取得到所述多个第二样本图像的图像信息，所述多个第二样本图像的图像信息至少包括所述目标人体组织对应的多个目标区域的分布信息；
    当获取到待分割的第一图像时，所述计算机设备调用所述图像分割模型，由所述图像分割模型根据所述图像信息，对所述第一图像进行分割，输出第二图像。
  2. 根据权利要求1所述的方法，其特征在于，所述第一初始模型、所述第二初始模型和所述图像分割模型均包括第一图像分割模块和第二图像分割模块，所述第一图像分割模块和第二图像分割模块分别对应于一种分割算法，所述第一图像分割模块用于对三维图像进行分割，所述第二图像分割模块用于对二维图像进行分割；
    相应地，由所述图像分割模型根据所述图像信息，对所述第一图像进行分割，输出第二图像，包括：
    所述计算机设备基于所述图像分割模型中的第一图像分割模块和第二图像分割模块中至少一个模块，以及所述图像信息，对所述第一图像进行分割，输出第二图像。
  3. 根据权利要求2所述的方法，其特征在于，所述基于所述图像分割模型中的第一图像分割模块和第二图像分割模块中至少一个模块，以及所述图像信息，对所述第一图像进行分割，输出第二图像，包括：
    所述计算机设备根据所述第一图像的属性信息，对所述第一图像进行预处理；
    所述计算机设备将预处理后的第一图像输入所述第一图像分割模块和第二 图像分割模块中至少一个模块中，由所述第一图像分割模块和所述第二图像分割模块中至少一个模块对所述第一图像进行分割，得到第三图像；
    所述计算机设备基于所述图像信息，对所述第三图像进行后处理，输出第二图像。
  4. 根据权利要求3所述的方法，其特征在于，所述计算机设备根据所述第一图像的属性信息，对所述第一图像进行预处理，包括：
    当根据所述属性信息确定所述第一图像中存在异常像素点时，所述计算机设备将所述异常像素点删除；
    当根据所述属性信息确定删除异常像素点后的第一图像的灰度范围大于目标范围时，所述计算机设备对所述第一图像进行规范化处理，将所述第一图像的灰度范围调整为目标范围内；
    当根据所述属性信息确定所述第一图像的通道数量大于一时，所述计算机设备将所述第一图像的每个像素值均减去目标图像均值；
    当根据所述属性信息确定所述第一图像的模态数量大于一时，所述计算机设备将所述第一图像输入模态融合模块，由所述模态融合模块从所述第一图像的多个像素值进行筛选，得到预处理后的第一图像的目标数量的像素值，所述预处理后的第一图像的模态数量为一。
  5. 根据权利要求3所述的方法，其特征在于，所述计算机设备基于所述图像信息，对所述第三图像进行后处理，输出第二图像，包括：
    所述计算机设备基于所述第三图像中的多个目标区域和所述图像信息所指示的所述多个目标区域的分布信息，对所述第三图像进行后处理，得到第二图像，目标区域为所述第三图像中目标类型的像素点所在区域，所述第二图像中多个目标区域的分布类型、目标区域的数量和目标区域的大小范围与所述多个目标区域的分布信息相同。
  6. 根据权利要求5所述的方法，其特征在于，所述计算机设备基于所述第三图像中的多个目标区域和所述图像信息所指示的所述多个目标区域的分布信息，对所述第三图像进行后处理，包括：
    当所述第三图像中目标区域的数量或大小范围与所述图像信息所指示的所 述多个目标区域的数量或大小范围不同时，所述计算机设备将所述第三图像中不符合所述多个目标区域的数量或大小范围不同的部分滤除。
  7. 根据权利要求5所述的方法，其特征在于，所述计算机设备基于所述第三图像中的多个目标区域和所述图像信息所指示的所述多个目标区域的分布信息，对所述第三图像进行后处理，包括：
    当任一目标区域内部存在背景像素点时，所述计算机设备将所述背景像素点更改为所述目标区域对应的目标类型的像素点。
  8. 根据权利要求3所述的方法，其特征在于，所述由所述第一图像分割模块和所述第二图像分割模块中至少一个模块对所述第一图像进行分割，得到第三图像，包括：
    所述计算机设备基于所述图像分割模型的模块选择参数，由所述第一图像分割模块，对所述第一图像进行分割，得到第一分割结果，基于所述第一分割结果，得到第三图像，所述第一分割结果用于指示所述第一图像的每个像素点为至少两个类型中每个类型的概率。
  9. 根据权利要求3所述的方法，其特征在于，所述由所述第一图像分割模块和所述第二图像分割模块中至少一个模块对所述第一图像进行分割，得到第三图像，包括：
    所述计算机设备基于所述图像分割模型的模块选择参数，由所述第二图像分割模块，对所述第一图像进行分割，得到第二分割结果，基于所述第二分割结果，得到第三图像，所述第二分割结果用于指示所述第一图像的每个像素点为至少两个类型中每个类型的概率。
  10. 根据权利要求3所述的方法，其特征在于，所述由所述第一图像分割模块和所述第二图像分割模块中至少一个模块对所述第一图像进行分割，得到第三图像，包括：
    所述计算机设备基于所述图像分割模型的模块选择参数，分别由所述第一图像分割模块和第二图像分割模块对所述第一图像进行分割，得到第一分割结果和第二分割结果，基于所述第一分割结果和第二分割结果，得到第三图像。
  11. 根据权利要求2所述的方法，其特征在于，所述第一图像分割模块对所述第一图像的分割过程包括：
    所述计算机设备基于所述第一图像分割模块的模块参数，对所述第一图像进行两次分类，得到第一分割结果，所述两次分类中第一次分类的分类对象为所述第一图像的所有像素点，第二次分类的分类对象为所述第一次分类结果中的前景像素点；
    所述第二图像分割模块对所述第一图像的分割过程包括：
    所述计算机设备基于所述第二图像分割模块的模块参数，对所述第一图像进行特征提取；
    所述计算机设备基于提取的特征，对所述第一图像的每个像素点进行分类，得到第二分割结果。
  12. 根据权利要求2所述的方法，其特征在于，所述计算机设备基于多个第二样本图像，对所述第二初始模型进行训练的过程包括：
    所述计算机设备基于所述多个第二样本图像对所述第二初始模型中的所述第一图像分割模块、所述第二图像分割模块进行训练，直到达到第一迭代停止次数时停止，得到第一图像分割模块和第二图像分割模块的模块参数；
    所述计算机设备基于所述多个第二样本图像和训练得到的所述第一图像分割模块和所述第二图像分割模块，对所述第二初始模型中的模块选择参数进行训练，直到达到第二迭代停止次数时停止，得到图像分割模型，所述模块选择参数用于决策选择所述第一图像分割模块和所述第二图像分割模块中至少一个分割模块对第一图像进行分割。
  13. 根据权利要求2所述的方法，其特征在于，所述图像信息还包括每个第二样本图像的属性信息；
    相应地，所述计算机设备基于多个第二样本图像，对所述第二初始模型进行训练的过程还包括：
    所述计算机设备基于所述每个第二样本图像的属性信息，对每个第二样本图像进行预处理，将预处理后的多个第二样本图像输入所述第一图像分割模块和所述第二图像分割模块中。
  14. 根据权利要求2所述的方法，其特征在于，所述图像信息还包括所述多个第二样本图像的图像数量；
    相应地，所述计算机设备基于多个第二样本图像，对所述第二初始模型进行训练的过程还包括：
    所述计算机设备基于所述图像数量，获取所述图像数量对应的图像分割子模块作为所述第二图像分割模块进行训练，所述第二图像分割模块中包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
  15. 一种图像分割装置，其特征在于，所述装置包括：
    训练模块，用于基于多个第一样本图像，对第一初始模型进行预训练，得到第二初始模型，所述多个第一样本图像包括多种人体组织图像，所述第二初始模型中包括所述多种人体组织对应的多个目标区域的分布信息；
    所述训练模块，还用于基于多个第二样本图像，对所述第二初始模型进行训练，得到图像分割模型，所述多个第二样本图像为目标人体组织图像，训练过程中所述图像分割模型获取得到所述多个第二样本图像的图像信息，所述多个第二样本图像的图像信息至少包括所述目标人体组织对应的多个目标区域的分布信息；
    分割模块，用于当获取到待分割的第一图像时，调用所述图像分割模型，由所述图像分割模型根据所述图像信息，对所述第一图像进行分割，输出第二图像。
  16. 根据权利要求15所述的装置，其特征在于，所述第一初始模型、所述第二初始模型和所述图像分割模型均包括第一图像分割模块和第二图像分割模块，所述第一图像分割模块和第二图像分割模块分别对应于一种分割算法，所述第一图像分割模块用于对三维图像进行分割，所述第二图像分割模块用于对二维图像进行分割；所述分割模块还用于基于所述图像分割模型中的第一图像分割模块和第二图像分割模块中至少一个模块，以及所述图像信息，对所述第一图像进行分割，输出第二图像。
  17. 根据权利要求16所述的装置，其特征在于，所述分割模块还用于根据所述第一图像的属性信息，对所述第一图像进行预处理；将预处理后的第一图像输入所述第一图像分割模块和第二图像分割模块中至少一个模块中，由所述第一图像分割模块和所述第二图像分割模块中至少一个模块对所述第一图像进行分割，得到第三图像；基于所述图像信息，对所述第三图像进行后处理，输出第二图像。
  18. 根据权利要求17所述的装置，其特征在于，所述分割模块还用于当根据所述属性信息确定所述第一图像中存在异常像素点时，将所述异常像素点删除；当根据所述属性信息确定删除异常像素点后的第一图像的灰度范围大于目标范围时，对所述第一图像进行规范化处理，将所述第一图像的灰度范围调整为目标范围内；当根据所述属性信息确定所述第一图像的通道数量大于一时，将所述第一图像的每个像素值均减去目标图像均值；当根据所述属性信息确定所述第一图像的模态数量大于一时，将所述第一图像输入模态融合模块，由所述模态融合模块从所述第一图像的多个像素值进行筛选，得到预处理后的第一图像的目标数量的像素值，所述预处理后的第一图像的模态数量为一。
  19. 根据权利要求17所述的装置，其特征在于，所述分割模块还用于基于所述第三图像中的多个目标区域和所述图像信息所指示的所述多个目标区域的分布信息，对所述第三图像进行后处理，得到第二图像，目标区域为所述第三图像中目标类型的像素点所在区域，所述第二图像中多个目标区域的分布类型、目标区域的数量和目标区域的大小范围与所述多个目标区域的分布信息相同。
  20. 根据权利要求19所述的装置，其特征在于，所述分割模块还用于当所述第三图像中目标区域的数量或大小范围与所述图像信息所指示的所述多个目标区域的数量或大小范围不同时，将所述第三图像中不符合所述多个目标区域的数量或大小范围不同的部分滤除。
  21. 根据权利要求19所述的装置，其特征在于，所述分割模块还用于当任一目标区域内部存在背景像素点时，将所述背景像素点更改为所述目标区域对应的目标类型的像素点。
  22. 根据权利要求17所述的装置，其特征在于，所述分割模块还用于基于所述图像分割模型的模块选择参数，由所述第一图像分割模块，对所述第一图像进行分割，得到第一分割结果，基于所述第一分割结果，得到第三图像，所述第一分割结果用于指示所述第一图像的每个像素点为至少两个类型中每个类型的概率。
  23. 根据权利要求17所述的装置，其特征在于，所述分割模块还用于基于所述图像分割模型的模块选择参数，由所述第二图像分割模块，对所述第一图像进行分割，得到第二分割结果，基于所述第二分割结果，得到第三图像，所述第二分割结果用于指示所述第一图像的每个像素点为至少两个类型中每个类型的概率。
  24. 根据权利要求17所述的装置，其特征在于，所述分割模块还用于基于所述图像分割模型的模块选择参数，分别由所述第一图像分割模块和第二图像分割模块对所述第一图像进行分割，得到第一分割结果和第二分割结果，基于所述第一分割结果和第二分割结果，得到第三图像。
  25. 根据权利要求16所述的装置，其特征在于，所述分割模块还用于基于所述第一图像分割模块的模块参数，对所述第一图像进行两次分类，得到第一分割结果，所述两次分类中第一次分类的分类对象为所述第一图像的所有像素点，第二次分类的分类对象为所述第一次分类结果中的前景像素点；基于所述第二图像分割模块的模块参数，对所述第一图像进行特征提取；基于提取的特征，对所述第一图像的每个像素点进行分类，得到第二分割结果。
  26. 根据权利要求16所述的装置，其特征在于，所述训练模块还用于基于所述多个第二样本图像对所述第二初始模型中的所述第一图像分割模块、所述第二图像分割模块进行训练，直到达到第一迭代停止次数时停止，得到第一图像分割模块和第二图像分割模块的模块参数；基于所述多个第二样本图像和训练得到的所述第一图像分割模块和所述第二图像分割模块，对所述第二初始模型中的模块选择参数进行训练，直到达到第二迭代停止次数时停止，得到图像 分割模型，所述模块选择参数用于决策选择所述第一图像分割模块和所述第二图像分割模块中至少一个分割模块对第一图像进行分割。
  27. 根据权利要求16所述的装置，其特征在于，所述图像信息还包括每个第二样本图像的属性信息，所述训练模块还用于基于所述每个第二样本图像的属性信息，对每个第二样本图像进行预处理，将预处理后的多个第二样本图像输入所述第一图像分割模块和所述第二图像分割模块中。
  28. 根据权利要求16所述的装置，其特征在于，所述图像信息还包括所述多个第二样本图像的图像数量，所述训练模块还用于基于所述图像数量，获取所述图像数量对应的图像分割子模块作为所述第二图像分割模块进行训练，所述第二图像分割模块中包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
  29. 一种计算机设备，，所述计算机设备包括处理器和存储器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如权利要求1至14任一项所述的方法。
  30. 一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如权利要求1至14任一项所述的方法。
PCT/CN2019/110541 2018-10-16 2019-10-11 图像分割方法、装置、计算机设备及存储介质 WO2020078263A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP19874073.0A EP3869456A4 (en) 2018-10-16 2019-10-11 IMAGE SEGMENTATION PROCESS AND APPARATUS, COMPUTER DEVICE AND RECORDING MEDIA
JP2021513795A JP7085062B2 (ja) 2018-10-16 2019-10-11 画像セグメンテーション方法、装置、コンピュータ機器およびコンピュータプログラム
KR1020217005619A KR102597385B1 (ko) 2018-10-16 2019-10-11 이미지 세그먼트화 방법 및 장치, 컴퓨터 디바이스 및 저장 매체
US17/173,172 US12002212B2 (en) 2018-10-16 2021-02-10 Image segmentation method and apparatus, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811205146.4 2018-10-16
CN201811205146.4A CN109410220B (zh) 2018-10-16 2018-10-16 图像分割方法、装置、计算机设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/173,172 Continuation US12002212B2 (en) 2018-10-16 2021-02-10 Image segmentation method and apparatus, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2020078263A1 true WO2020078263A1 (zh) 2020-04-23

Family

ID=65467353

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110541 WO2020078263A1 (zh) 2018-10-16 2019-10-11 图像分割方法、装置、计算机设备及存储介质

Country Status (6)

Country Link
US (1) US12002212B2 (zh)
EP (1) EP3869456A4 (zh)
JP (1) JP7085062B2 (zh)
KR (1) KR102597385B1 (zh)
CN (2) CN109410220B (zh)
WO (1) WO2020078263A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689363A (zh) * 2021-09-07 2021-11-23 北京顺势兄弟科技有限公司 一种人像图处理方法、装置、电子设备、存储介质
CN114373162A (zh) * 2021-12-21 2022-04-19 国网江苏省电力有限公司南通供电分公司 用于变电站视频监控的危险区域人员入侵检测方法及系统

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109410220B (zh) * 2018-10-16 2019-12-24 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质
CN109993758A (zh) * 2019-04-23 2019-07-09 北京华力兴科技发展有限责任公司 分割方法、分割装置、计算机设备及存储介质
CN110189341B (zh) * 2019-06-05 2021-08-10 北京青燕祥云科技有限公司 一种图像分割模型训练的方法、图像分割的方法及装置
CN110472623B (zh) * 2019-06-29 2022-08-09 华为技术有限公司 图像检测方法、设备以及系统
CN110334679B (zh) * 2019-07-11 2021-11-26 厦门美图之家科技有限公司 人脸点处理方法及装置
CN110363168A (zh) * 2019-07-19 2019-10-22 山东浪潮人工智能研究院有限公司 一种基于卷积神经网络的三维立体图识别系统
CN110555833B (zh) * 2019-08-30 2023-03-21 联想(北京)有限公司 图像处理方法、装置、电子设备以及介质
CN110599492B (zh) * 2019-09-19 2024-02-06 腾讯科技（深圳）有限公司 图像分割模型的训练方法、装置、电子设备及存储介质
CN110866925B (zh) * 2019-10-18 2023-05-26 拜耳股份有限公司 一种用于图像分割的方法和装置
JP7410619B2 (ja) * 2019-10-31 2024-01-10 キヤノン株式会社 画像処理装置、画像処理方法およびプログラム
CN111091140B (zh) * 2019-11-20 2024-04-02 南京旷云科技有限公司 目标分类方法、装置及可读存储介质
CN110991298B (zh) * 2019-11-26 2023-07-14 腾讯科技（深圳）有限公司 图像的处理方法和装置、存储介质及电子装置
CN111539961A (zh) * 2019-12-13 2020-08-14 山东浪潮人工智能研究院有限公司 一种目标分割方法、装置以及设备
CN111402268B (zh) * 2020-03-16 2023-05-23 苏州科技大学 一种医学图像中肝脏及其病灶分割的方法
CN111524204B (zh) * 2020-05-06 2023-06-27 杭州小影创新科技股份有限公司 一种人像发丝动漫化纹理生成方法
CN111598160B (zh) * 2020-05-14 2023-04-07 腾讯科技（深圳）有限公司 图像分类模型的训练方法、装置、计算机设备及存储介质
CN112258456B (zh) * 2020-09-28 2023-10-17 汕头大学 一种基于卷积神经网络监督的三维图像分割方法
CN112233128B (zh) * 2020-10-15 2021-11-02 推想医疗科技股份有限公司 图像分割方法及模型的训练方法、装置、介质、电子设备
CN112529911B (zh) * 2020-12-07 2024-02-09 重庆大学 一种胰腺图像分割模型的训练方法、图像分割方法和装置
CN112508974B (zh) * 2020-12-14 2024-06-11 北京达佳互联信息技术有限公司 图像分割模型的训练方法、装置、电子设备和存储介质
CN113159159B (zh) * 2021-04-15 2023-09-29 东北大学 一种基于改进cnn的小样本图像分类方法
CN113689436B (zh) * 2021-09-29 2024-02-02 平安科技（深圳）有限公司 图像语义分割方法、装置、设备及存储介质
CN114648514B (zh) * 2022-03-30 2022-11-29 中国人民解放军总医院第二医学中心 一种脑动脉定位提取方法、装置、电子设备及存储介质
CN114926471B (zh) * 2022-05-24 2023-03-28 北京医准智能科技有限公司 一种图像分割方法、装置、电子设备及存储介质
CN114820584B (zh) * 2022-05-27 2023-02-21 北京安德医智科技有限公司 肺部病灶定位装置
CN115205246B (zh) * 2022-07-14 2024-04-09 中国南方电网有限责任公司超高压输电公司广州局 换流阀电晕放电紫外图像特征提取方法和装置
CN115393685B (zh) * 2022-10-28 2023-03-28 良业科技集团股份有限公司 基于可拓展模型的文旅数据处理方法及系统
CN115908457B (zh) * 2023-01-06 2023-05-23 脑玺(苏州)智能科技有限公司 低密度梗死区分割方法、分析方法、装置、系统、设备及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160063720A1 (en) * 2014-09-02 2016-03-03 Impac Medical Systems, Inc. Systems and methods for segmenting medical images based on anatomical landmark-based features
CN105608479A (zh) * 2016-03-01 2016-05-25 北京正安维视科技股份有限公司 结合深度数据的异常行为检测方法及系统
CN107240102A (zh) * 2017-04-20 2017-10-10 合肥工业大学 基于深度学习算法的恶性肿瘤计算机辅助早期诊断方法
CN107967491A (zh) * 2017-12-14 2018-04-27 北京木业邦科技有限公司 木板识别的机器再学习方法、装置、电子设备及存储介质
CN108597053A (zh) * 2018-04-25 2018-09-28 北京御航智能科技有限公司 基于图像数据和神经网络的杆塔和通道目标识别与缺陷诊断方法
CN109410220A (zh) * 2018-10-16 2019-03-01 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8831703B2 (en) * 2006-10-23 2014-09-09 The General Hospital Corporation Selective MR imaging of segmented anatomy
US8936909B2 (en) * 2008-07-18 2015-01-20 Qiagen Gmbh Method for determining the origin of a sample
US20130210648A1 (en) * 2009-01-14 2013-08-15 The United States of America, as Represented by the Secretary, Department of Health and Human Services Ratio based biomarkers and methods of use thereof
US9589360B2 (en) * 2012-07-23 2017-03-07 General Electric Company Biological unit segmentation with ranking based on similarity applying a geometric shape and scale model
WO2016086744A1 (en) * 2014-12-02 2016-06-09 Shanghai United Imaging Healthcare Co., Ltd. A method and system for image processing
CN104809730B (zh) * 2015-05-05 2017-10-03 上海联影医疗科技有限公司 从胸部ct图像提取气管的方法和装置
WO2016092408A1 (en) * 2014-12-09 2016-06-16 Koninklijke Philips N.V. Feedback for multi-modality auto-registration
CN104537676B (zh) 2015-01-12 2017-03-22 南京大学 一种基于在线学习的渐进式图像分割方法
WO2017027475A1 (en) * 2015-08-07 2017-02-16 Jianming Liang Methods, systems, and media for simultaneously monitoring colonoscopic video quality and detecting polyps in colonoscopy
US10368956B2 (en) * 2016-02-16 2019-08-06 The Johns Hopkins University MR-levelcheck-2: method for localization of structures in projection images
US10420523B2 (en) * 2016-03-21 2019-09-24 The Board Of Trustees Of The Leland Stanford Junior University Adaptive local window-based methods for characterizing features of interest in digital images and systems for practicing same
EP3264365A1 (en) * 2016-06-28 2018-01-03 Siemens Healthcare GmbH Method and device for registration of a first image data set and a second image data set of a target region of a patient
US20180018757A1 (en) * 2016-07-13 2018-01-18 Kenji Suzuki Transforming projection data in tomography by means of machine learning
CN109690554B (zh) * 2016-07-21 2023-12-05 西门子保健有限责任公司 用于基于人工智能的医学图像分割的方法和系统
US9947102B2 (en) * 2016-08-26 2018-04-17 Elekta, Inc. Image segmentation using neural network method
US9943225B1 (en) * 2016-09-23 2018-04-17 International Business Machines Corporation Early prediction of age related macular degeneration by image reconstruction
CN107665351B (zh) * 2017-05-06 2022-07-26 北京航空航天大学 基于难样本挖掘的机场检测方法
CN107644418B (zh) * 2017-09-26 2019-11-08 山东大学 基于卷积神经网络的视盘检测方法及系统
CN107563123A (zh) 2017-09-27 2018-01-09 百度在线网络技术（北京）有限公司 用于标注医学图像的方法和装置
EP3747013A4 (en) * 2018-01-23 2021-09-29 Spring Discovery, Inc. METHODS AND SYSTEMS FOR DETERMINING THE BIOLOGICAL AGE OF SAMPLES
CN108053417B (zh) * 2018-01-30 2019-12-17 浙江大学 一种基于混合粗分割特征的3D U-Net网络的肺分割装置
EP3818496A4 (en) * 2018-07-02 2022-04-20 Tempus Labs, Inc. 3D RADIOMICS PLATFORM FOR IMAGING BIOMARKER DEVELOPMENT
KR20210041587A (ko) * 2018-07-30 2021-04-15 젠셀랩, 엘엘씨 엑스레이 이미징 시스템 및 방법, 그리고 조영제

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160063720A1 (en) * 2014-09-02 2016-03-03 Impac Medical Systems, Inc. Systems and methods for segmenting medical images based on anatomical landmark-based features
CN105608479A (zh) * 2016-03-01 2016-05-25 北京正安维视科技股份有限公司 结合深度数据的异常行为检测方法及系统
CN107240102A (zh) * 2017-04-20 2017-10-10 合肥工业大学 基于深度学习算法的恶性肿瘤计算机辅助早期诊断方法
CN107967491A (zh) * 2017-12-14 2018-04-27 北京木业邦科技有限公司 木板识别的机器再学习方法、装置、电子设备及存储介质
CN108597053A (zh) * 2018-04-25 2018-09-28 北京御航智能科技有限公司 基于图像数据和神经网络的杆塔和通道目标识别与缺陷诊断方法
CN109410220A (zh) * 2018-10-16 2019-03-01 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3869456A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689363A (zh) * 2021-09-07 2021-11-23 北京顺势兄弟科技有限公司 一种人像图处理方法、装置、电子设备、存储介质
CN113689363B (zh) * 2021-09-07 2024-03-29 北京顺势兄弟科技有限公司 一种人像图处理方法、装置、电子设备、存储介质
CN114373162A (zh) * 2021-12-21 2022-04-19 国网江苏省电力有限公司南通供电分公司 用于变电站视频监控的危险区域人员入侵检测方法及系统
CN114373162B (zh) * 2021-12-21 2023-12-26 国网江苏省电力有限公司南通供电分公司 用于变电站视频监控的危险区域人员入侵检测方法及系统

Also Published As

Publication number Publication date
US20210166395A1 (en) 2021-06-03
EP3869456A1 (en) 2021-08-25
CN109410220B (zh) 2019-12-24
EP3869456A4 (en) 2021-12-22
JP2022500754A (ja) 2022-01-04
CN111062952B (zh) 2022-09-30
CN111062952A (zh) 2020-04-24
KR102597385B1 (ko) 2023-11-03
US12002212B2 (en) 2024-06-04
CN109410220A (zh) 2019-03-01
JP7085062B2 (ja) 2022-06-15
KR20210038622A (ko) 2021-04-07

Similar Documents

Publication Publication Date Title
WO2020078263A1 (zh) 图像分割方法、装置、计算机设备及存储介质
US11403763B2 (en) Image segmentation method and apparatus, computer device, and storage medium
CN110348543B (zh) 眼底图像识别方法、装置、计算机设备及存储介质
US20210011942A1 (en) Method, apparatus, and storage medium for classifying multimedia resource
CN108594997B (zh) 手势骨架构建方法、装置、设备及存储介质
WO2019105285A1 (zh) 人脸属性识别方法、电子设备及存储介质
CN111091576A (zh) 图像分割方法、装置、设备及存储介质
CN110555839A (zh) 缺陷检测识别方法、装置、计算机设备及存储介质
CN110650379B (zh) 视频摘要生成方法、装置、电子设备及存储介质
CN110570460B (zh) 目标跟踪方法、装置、计算机设备及计算机可读存储介质
CN110807361A (zh) 人体识别方法、装置、计算机设备及存储介质
US20220309836A1 (en) Ai-based face recognition method and apparatus, device, and medium
CN108830186B (zh) 文本图像的内容提取方法、装置、设备及存储介质
CN112749613B (zh) 视频数据处理方法、装置、计算机设备及存储介质
CN111192262A (zh) 基于人工智能的产品缺陷分类方法、装置、设备及介质
CN112036331A (zh) 活体检测模型的训练方法、装置、设备及存储介质
CN113706447A (zh) 脑中线识别方法、装置、设备及存储介质
CN113570645A (zh) 图像配准方法、装置、计算机设备及介质
CN111598896A (zh) 图像检测方法、装置、设备及存储介质
KR20180097912A (ko) 사용자 단말을 이용하여 사용자의 감정을 분석하는 감정 분석 방법 및 시스템
CN113516665A (zh) 图像分割模型的训练方法、图像分割方法、装置、设备
CN110163192B (zh) 字符识别方法、装置及可读介质
CN113192072B (zh) 图像分割方法、装置、设备及存储介质
CN111639639A (zh) 检测文本区域的方法、装置、设备及存储介质
CN113743186A (zh) 医学图像的处理方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19874073

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20217005619

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021513795

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019874073

Country of ref document: EP

Effective date: 20210517