WO2020078268A1 - 图像分割方法、装置、计算机设备及存储介质 - Google Patents

图像分割方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2020078268A1
WO2020078268A1 PCT/CN2019/110557 CN2019110557W WO2020078268A1 WO 2020078268 A1 WO2020078268 A1 WO 2020078268A1 CN 2019110557 W CN2019110557 W CN 2019110557W WO 2020078268 A1 WO2020078268 A1 WO 2020078268A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image segmentation
segmentation
module
model
Prior art date
Application number
PCT/CN2019/110557
Other languages
English (en)
French (fr)
Inventor
陈思宏
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP19873499.8A priority Critical patent/EP3869455A4/en
Publication of WO2020078268A1 publication Critical patent/WO2020078268A1/zh
Priority to US17/039,925 priority patent/US11403763B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present application relates to the field of computer technology, and in particular, to an image segmentation method, device, computer equipment, and storage medium.
  • image segmentation technology is more and more widely used, for example, medical image segmentation, natural image segmentation and so on.
  • the image segmentation technology refers to the technology that divides the image into several specific areas with unique properties and proposes the target of interest.
  • Human tissue usually has a fixed range limit and morphological distribution characteristics. Designing a specific image segmentation model according to different tissue distribution conditions can effectively improve the accuracy of segmentation.
  • a distribution of human tissue is nested, that is, there is another tissue within one tissue of the human body.
  • a brain tumor is a nested tissue. From the nested outer layer to the inner layer, there are edema, non-enhanced tumor, and enhanced tumor.
  • image segmentation methods generally use Cascaded Anisotropic Convolutional Neural Networks (Cascaded ACNN), which is used to segment brain tumors.
  • Cascaded ACNN Cascaded Anisotropic Convolutional Neural Networks
  • the network includes three different types of segmentation models: WNET (Whole Tumor Network), TNET (Tumor Network) and ENET (Enhancing Tumor Network).
  • WNET Whole Tumor Network
  • TNET Tumor Network
  • ENET Endhancing Tumor Network
  • the image segmentation method is usually to input the image into the network, WNET segmentation of the complete image to get the edema area of the image, input the image of the edema area into TNET, TNET segment the image of the edema area, get the non- The area of the enhanced tumor, enter the area of the non-enhanced tumor into the ENET, ENET divides the area of the non-enhanced tumor, and obtains the area of the enhanced tumor, and finally Cascaded ACNN divides the area obtained by the three models from small to large The sequences are overlapped together to obtain the segmented image.
  • the above Cascaded ACNN is only suitable for segmentation of brain tumors. If you need to segment other nested human tissue images, you need to redesign the model based on the distribution of other nested human tissue images.
  • the training is performed directly on the basis of, so the above image segmentation method is poor in versatility, applicability and practicability.
  • an image segmentation method includes:
  • Computer equipment acquires multiple sample images
  • the computer device calls the initial model, inputs the multiple sample images into the initial model, and trains the initial model based on the multiple sample images to obtain an image segmentation model.
  • the initial model is used to The number of pixel types of multiple sample images determines the number of image segmentation modules, and different image segmentation modules are used to segment different regions of the image;
  • the computer device calls the image segmentation model, and the image segmentation model splits the first image based on multiple image segmentation modules and outputs the second image.
  • the plurality of sample images and the first image are all target human tissue images.
  • an image segmentation device including:
  • Acquisition module for acquiring multiple sample images
  • the training module is used to call the initial model, input the multiple sample images into the initial model, and train the initial model based on the multiple sample images to obtain an image segmentation model.
  • the initial model is used to The number of types of pixel points of the multiple sample images determines the number of image segmentation modules. Different image segmentation modules are used to segment different regions of the image;
  • the segmentation module is used to call the image segmentation model when the first image to be segmented is acquired, and the image segmentation model splits the first image based on multiple image segmentation modules and outputs the second image, Both the plurality of sample images and the first image are target human tissue images.
  • a computer device in one aspect, includes a processor and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor Perform the method described in the above embodiment.
  • one or more non-volatile storage media storing computer-readable instructions are provided.
  • the computer-readable instructions are executed by one or more processors, the one or more processors execute the foregoing embodiments The method.
  • FIG. 1 is a schematic diagram of a Cascaded ACNN segmentation process provided by the background technology of this application;
  • FIG. 3 is a flowchart of an image segmentation model training method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a stream segmentation process provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an image cropping method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an image cropping method provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an image segmentation model provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a fusion method of segmentation results provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of an image segmentation method provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an image segmentation device according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • FIG. 2 is an implementation environment of an image segmentation method provided by an embodiment of the present application.
  • the implementation environment may include multiple computer devices.
  • the multiple computer devices may implement data interaction through a wired connection, or may implement data interaction through a wireless network connection, which is not limited in the embodiments of the present application.
  • the computer device 201 may be used to segment the image.
  • the image may be a medical image, that is, a human tissue image, and the distribution of the human tissue is nested, that is, There is another tissue within one tissue in the human tissue image.
  • the image segmentation method can be applied to any nested tissue analysis scenario, for example, liver and liver cancer analysis, pancreas and pancreas cancer analysis, lung and lung cancer analysis, brain tumor analysis, or brain cancer analysis scenario.
  • the image segmentation method can also be applied to other human tissue image segmentation scenarios, and the embodiments of the present application are not enumerated here one by one.
  • the image can also be other types of images, and the image segmentation method can also be applied to other image segmentation scenes, for example, landscape image segmentation scenes.
  • the computer device 202 can be used to collect images and send the collected images to the computer device 201, and the computer device 201 provides an image segmentation service.
  • the computer device 201 can also collect images and segment the collected images, which is not limited in the embodiments of the present application.
  • the computer device 202 may also be used to store images obtained from other computer devices, and the computer device 201 may obtain the stored images from the computer device 202 for segmentation.
  • both the computer device 201 and the computer device 202 can be provided as a terminal or a server, which is not limited in the embodiments of the present application.
  • FIG. 3 is a flowchart of an image segmentation model training method provided by an embodiment of the present application.
  • the image segmentation model training method can be applied to a computer device, which can be the computer device 201 in the above implementation environment, or Other computer equipment. That is, the image segmentation model can be trained on the computer device 201, or the image segmentation model can be trained on other computer devices, the image segmentation model is processed into a configuration file, and the configuration file is sent to the computer device 201. Then, the computer device 201 stores an image segmentation model.
  • the computer device 201 may also call an image segmentation model trained on other computer devices when there is an image segmentation requirement, which is not limited in this embodiment of the present application. Referring to FIG. 3, the method may include the following steps:
  • the computer device acquires a plurality of sample images, and each sample image carries a label, which is used to indicate the target segmentation result of the sample image.
  • the multiple sample images are target human tissue images.
  • the computer device can train the initial model based on the multiple sample images to obtain an image segmentation model.
  • the image segmentation model thus obtained can segment the target human tissue images.
  • the target human tissue image may be the image of the nested tissue, for example, liver image, pancreas image, lung image, brain image, etc.
  • the plurality of sample images may be stored in the computer device, and when the image segmentation model training is required, the computer device may obtain the stored plurality of sample images.
  • each sample image may also carry a label indicating the target segmentation result, where the target segmentation result refers to the correct segmentation result of the sample image, or refers to the true segmentation result of the sample image. In this way, during the model training process, you can know whether the training model accurately divides the sample image, and whether you need to continue training the model, so that the trained model can obtain the target segmentation result when segmenting the sample image, or Very close to the target segmentation result.
  • the multiple sample images may also be stored in other computer equipment, and the computer equipment may be obtained from other computer equipment when the image segmentation model training is required, which is not limited in the embodiments of the present application.
  • the multiple sample images can be stored in an image database, and each sample image also carries a label. Then, this step 301 can obtain multiple sample images from the image database for the computer device.
  • the computer device calls the initial model, and inputs the plurality of sample images into the initial model.
  • the computer device After the computer device acquires multiple sample images, it can call the initial model, train the initial model based on the multiple sample images, and obtain an image segmentation model, so that the subsequent first image can be accurately segmented based on the image segmentation model .
  • the initial model may be stored in the computer device, and the stored initial model is directly called in step 302. In some embodiments, the initial model may not be stored in the computer device.
  • the initial model is called from other computer devices and the subsequent model training process is performed.
  • the embodiments of the present application specifically adopt There is no limitation on which way of implementation.
  • model parameters of the initial model are initial values
  • the computer device can use the multiple sample images as training samples and verification samples to train the initial model, that is, adjust the model parameters of the initial model through the sample images , So that the model parameters after multiple adjustments can obtain more accurate segmentation results when segmenting the first image.
  • the computer device inputs the multiple sample images into the initial model, and each sample image can be segmented by the initial model. Based on the segmentation result of the initial model and the label of the sample image, that is, the target segmentation result of the sample image, the determination The segmentation ability of the initial model, so that the model parameters of the initial model can be adjusted to continuously improve the segmentation ability of the initial model, so that the image segmentation model obtained by subsequent training can be accurately segmented.
  • the model parameters of the initial model may be obtained by pre-training based on multiple sample images, and the multiple sample images may include multiple human tissue images, and the multiple human tissues are all
  • this initial model obtains a priori knowledge through the pre-training process, so that the initial model has a certain understanding of the image of the nested organization, which can be used to simulate the rotation of medical students in various departments, Therefore, the medical student can possess certain medical knowledge or clinical knowledge.
  • the initial model obtained in this way can effectively reduce the number of model trainings during subsequent model training, and can also be used to segment the various nested organization images based on various nested organization images, which can effectively improve The practicability and versatility of the initial model and the image segmentation model trained based on the initial model.
  • the computer device may perform matching based on the acquired multiple sample images and the sample images in the historical processing data, obtain the similarity between the multiple sample images and the historical sample images, and then compare the sample with the largest similarity
  • the model parameters of the image segmentation model of the image are used as the model parameters of the above initial model, that is, the above initial value, so that considering that similar sample images may be processed in the historical processing data, considering the similarity of the sample images, when the sample image is segmented
  • the required parameters may be similar.
  • model parameters of the two initial models may also be preset by relevant technical personnel, which is not limited in the embodiments of the present application.
  • the initial model in the computer device determines the number of image segmentation modules according to the number of types of pixels of the plurality of sample images.
  • the initial model may adopt a stream segmentation scheme, that is, the sample images are sequentially segmented by multiple image segmentation modules, and for the two adjacent image segmentation modules, the previous image segmentation module segments the images After that, the original image can be cropped, and the cropped image can be input to the subsequent image segmentation module, so that the subsequent image segmentation module can continue segmentation based on the segmentation result of the previous image segmentation module.
  • the image is segmented multiple times progressively, and the focus is concentrated step by step, which realizes a combination of rough and meticulous segmentation, which makes the segmentation results more accurate and is also suitable for segmentation tasks of different difficulty.
  • one-step segmentation may be performed first, and then the region obtained by the segmentation may be segmented more finely.
  • This initial model can support the segmentation requirements of various nested organized images, and can support multiple types of segmentation. Due to different human tissue images, the number of pixel types needs to be determined during the segmentation process. For example, the brain tumor segmentation scene needs to determine the type of pixel points can include 4 types: background, edema, non-enhanced tumor and enhanced tumor. The liver cancer segmentation scene needs to determine the type of pixel points can include three types: background, liver and liver cancer. The initial model can determine the number of image segmentation modules based on this type of data. Therefore, different human tissue images can be trained using the initial model to meet the segmentation requirements, effectively improving the versatility and practicality of the initial model.
  • the step 303 may be: the initial model uses the number of types of foreground pixel points of the plurality of sample images as the number of image segmentation modules in the initial model.
  • the types of pixels may include at least two types, and the at least two types may include two types: one type is a background and the other type is a foreground, and the foreground is one or more types other than the background in the at least two types.
  • the corresponding pixels are background pixels and foreground pixels. That is, pixels of type background are background pixels, and pixels of type foreground are foreground pixels.
  • the initial model determines multiple image segmentation modules, each of which corresponds to a type of pixel, that is, each image segmentation module will focus on a type of pixel segmentation.
  • the human tissue image is a brain image
  • the number of types of foreground pixels is 3, that is, there are three types of foreground pixels: edema, non-enhanced tumor, and enhanced tumor.
  • the initial model can be determined
  • the number of image segmentation modules is 3.
  • the images are sequentially segmented by the three image segmentation modules, the first image segmentation module focuses on segmentation of edema regions, the second image segmentation module focuses on segmentation of non-enhanced tumor regions, and the third image segmentation module focuses on segmentation Enhance the tumor area to get the segmentation result of the image.
  • the initial model can directly obtain the number of types of foreground pixels of the sample image, or the number of types of pixels, and reduce the number of types of pixels by one to obtain the number of types of foreground pixels.
  • the embodiments of the present application do not limit the specific implementation manner.
  • the target segmentation result in the tags of the multiple sample images is used to indicate the type of each pixel of the sample image, then the type data of the pixels of the multiple sample images may be based on the The label gets.
  • the initial model of the computer device can also analyze the labels of the multiple sample images to obtain the number of types of pixels of the multiple sample images, so that step 303 can be executed based on the number of types To determine the number of image segmentation modules. For example, the initial model may count the number of types of pixels in the target segmentation result in the label, or may only count the number of types of pixels in the foreground.
  • the initial model in this embodiment of the present application can analyze the sample image by itself to determine the number of image segmentation modules.
  • the number of the image segmentation module is variable.
  • the initial model can determine the number of image segmentation modules by itself. Therefore, the initial model provided by the embodiments of the present application can be applied to the scene of segmenting a variety of human tissue images. better.
  • the multiple image segmentation modules in the initial model sequentially segment each sample image to obtain the segmentation result of each image segmentation module for the sample image.
  • the initial model can segment the sample images in sequence based on multiple image segmentation modules to obtain segmentation results. Specifically, for each image segmentation module, the input image may be subjected to feature extraction based on the module parameters to obtain the image features. Then, based on the extracted features, each pixel in the image can be classified to obtain the segmentation result.
  • the image segmentation module may include at least one image segmentation submodule, and different image segmentation submodules have different depths.
  • the image segmentation module can be implemented using a deep residual network (ResNet).
  • the image segmentation module may include two image segmentation sub-modules: ResNet-18 and ResNet-152, where the depth of ResNet-152 is greater than the depth of ResNet-18.
  • the initial model can also obtain an image segmentation submodule corresponding to the number of images as the image segmentation module according to the number of images of the plurality of sample images.
  • an appropriate image segmentation sub-module can be selected for training according to the number of images of the sample image, so that the problem of over-fitting or poor segmentation ability of the trained model can be avoided.
  • 2D two-dimensional
  • 3D three-dimensional three-dimensional
  • the basic network is not limited in the embodiments of the present application.
  • the initial model may also store a correspondence between the number of images and the image segmentation sub-module, and the initial model performs the selection step of the image segmentation sub-module based on the correspondence.
  • the greater the number of images the greater the depth of the acquired image segmentation sub-module. This can effectively deal with the situation of small data, and the model can be trained when the number of samples is small, to obtain an image segmentation model with good segmentation effect.
  • the step of acquiring the image segmentation module may be: when the number of images of the plurality of sample images is greater than a preset number, the initial model acquires the first image segmentation sub Module; when the number of images of the plurality of sample images is less than or equal to the preset number, the initial model obtains the second image segmentation sub-module.
  • the depth of the first image segmentation sub-module is greater than the depth of the second image segmentation sub-module.
  • the preset quantity may be preset by relevant technical personnel, and the specific value of the preset quantity is not limited in the embodiment of the present application.
  • the first image segmentation sub-module may be ResNet-152
  • the second image segmentation sub-module may be ResNet-18.
  • the acquisition step of the above image segmentation module may be: when the sample image When the number of images is less than or equal to 100, ResNet-18 can be used as the basic model, and when the number of images of the sample image is greater than 100, ResNet-101 can be used as the basic model.
  • each image segmentation module can be improved based on Unity Networking (unet). Due to its unique progressive upsampling and skip connection structure, unet is particularly suitable for fine structures in human tissue images Of division.
  • the sample images can be segmented sequentially based on the multiple image segmentation modules.
  • the initial model may segment the third image based on the first image segmentation module to obtain a first segmentation result.
  • the first image segmentation module is The image segmentation module in the order of the two adjacent image segmentation modules.
  • the initial model is based on the second image segmentation module to segment the fourth image cropped based on the first segmentation result to obtain a second segmentation result.
  • the second image segmentation module is the order in which the two adjacent image segmentation modules are After the image segmentation module, the fourth image is a partial area of the third image.
  • the first segmentation result and the second segmentation result are used to indicate the probability that each pixel of the image is each of at least two types.
  • the third image refers to an image input to the first image segmentation module
  • the fourth image is an image obtained by the first image segmentation module cropping the third image based on the first segmentation result.
  • the fourth image includes pixels of the first type indicated by the first segmentation result, and the first type is a type corresponding to the first image segmentation module.
  • the type corresponding to the second image segmentation module is the second type. Since the target human tissue is a nested tissue, the area where the pixels of the second type are located is inside the area where the pixels of the first type are located. It is possible to focus on segmenting the pixels of the first type first, and then focus on segmenting the pixels of the second type more closely in the area where the pixels of the first type are located. It should be noted that each image segmentation module can classify pixel points and determine the probability that the pixel points are of various types, instead of focusing only on the type corresponding to the module, but focusing more on the type corresponding to the segmentation module.
  • the first image segmentation module segments the third image, determines the probability that each pixel is of each type, and initially determines the type of each pixel with this probability.
  • the first image segmentation module focuses on the first type, Therefore, the fourth image including the area where the pixels of the first type are located can be input to the second image segmentation module, and the second image segmentation module can continue to segment the fourth image, focusing more on segmenting the second type. If the first image segmentation module is the first image segmentation module among the image segmentation modules of the initial model, the third image is the input sample image itself.
  • the initial model may also obtain the pixel ratio between adjacent target regions in the multiple target regions corresponding to the multiple sample images.
  • the target region is the The area where pixels of the target type are located in multiple sample images.
  • the image segmentation module may determine the pixel ratio between adjacent target regions in the multiple target regions and The size relationship of the target value, crop the image.
  • the target value refers to the threshold of the pixel ratio, which can be used to measure whether the pixel ratio between the target areas exceeds the user tolerance, that is, to measure whether the ratio between multiple target areas is unbalanced, the target value It can be pre-set by the relevant technical personnel, or the computer device can provide the user with setting options or input functions, obtain the value set by the user as the target value, or process the value set by the user, for example, the user can enter 3, the computer The device can take the reciprocal of 3 to obtain the target value of 1/3. Of course, 3 can also be used as the target value.
  • the specific value of the target value is not limited in the embodiment of the present application.
  • the cropping range may be determined based on the size of the target area corresponding to the type of the image segmentation module, for example, the target area corresponding to the current module may be used as the center , And expand by a certain percentage (for example, 10%) for cropping, so that the image input to the next module includes not only the target area determined by the current module, but also some pixel information around the target area, so that the next module can also This part of the area is segmented again to avoid errors due to inaccurate segmentation of a certain module and improve the accuracy of image segmentation.
  • a certain percentage for example, 10%
  • the initial model may also obtain the connected domain ranges of multiple target regions corresponding to the multiple sample images.
  • This step can be obtained by performing connected domain processing on the label of the sample image. In this way, by performing statistical processing on the sample image, a standard value is obtained, and the standard value is used to determine an appropriate cropping range.
  • the image segmentation module can determine the target pixel ratio between adjacent target regions in the multiple target regions and the target The relationship between the magnitude of the numerical values and the range of the connected domains of the multiple target regions crop the image.
  • this step 304 may be: For the first image segmentation module, the first image segmentation module according to the pixel ratio of the first target area and the second target area and the target value Size relationship and the connected domain range of the first target area or the second target area, the third image is cropped, the first target area is the area of the first type of pixel corresponding to the first image segmentation module The second target area is an area where pixels of the second type corresponding to the second image segmentation module are located.
  • the selection of the connected domain range according to the first target area or the second target area in the above process may be determined according to the above size range. Specifically, it may include two selection cases: Case 1. When the pixel ratio of the first target area and the second target area is less than the target value, the first image segmentation module is based on the range of the connected domain of the first target area. The third image is cropped. Case 2: When the pixel ratio of the first target area and the second target area is greater than or equal to the target value, the first image segmentation module crops the third image based on the connected domain range of the second target area.
  • the size of the cropped fourth image is based on the connected domain range and the first coefficient of the first target area get.
  • the size of the cropped fourth image is obtained based on the connected domain range of the second target area and the second coefficient .
  • the first coefficient is smaller than the second coefficient, the first coefficient and the second coefficient are greater than one.
  • both the first coefficient and the second coefficient can be preset by relevant technical personnel, and the value of the embodiment is not limited in the embodiment of the present application.
  • the first case is the case where the pixel ratio is less than the target value, which can be called the cropping method in the first case Is an all-inclusive crop.
  • the target value is 3, and the pixel ratio of the first target area and the second target area is 2, which is smaller than the target value, so that the first target area and the second target area
  • the size of the two target areas is not much different, that is, the number of pixels is relatively balanced, then the sample image is cropped so that the pixels of the second target area are unlikely to disappear, and the first is determined based on the first segmentation result.
  • the range obtained by the connected domain range (statistic value) of the first target area and the first coefficient is used as the cropping range. Therefore, the first target area in the third image can be used as the center to expand the first coefficient to obtain the fourth image input to the second image segmentation module.
  • the range of the first coefficient extended outside the connected domain can be used as the cropping range.
  • the height of the connected domain can be extended by 5% and the width by 5%.
  • the first coefficient may also be 110%, and the product of the connected domain range and the first coefficient may be used as the clipping range.
  • the three tissues are from the outer layer to the inner layer: edema (represented as edema), non-enhanced tumor (represented as active ), Enhanced tumor (represented as necrotic), CC represents the external label box, used to indicate the cropping range.
  • edema represented as edema
  • non-enhanced tumor represented as active
  • Enhanced tumor represented as necrotic
  • CC represents the external label box, used to indicate the cropping range.
  • the previous module outputs the corresponding type of target area as the center, and crops the image with the connected domain range of the target area, that is, expands the connected domain range by 10% as the input of the next layer model.
  • this module mainly divides edema, centering on edema CC1 in the prediction image, the edema connected domain range (connected domain range of the first target area) is expanded by 10% for the region cropped image, Get the input of the active model, and so on for the following modules.
  • the cropping range exceeds the range of the image input by the current module, it is completed with the full full image area, and if it exceeds the full image area, it is discarded.
  • case two is the case where the pixel ratio is greater than the target value, for example, the target value is 3, and the pixel ratio of the first target area and the second target area is 4, which is greater than the target value, so that the first target area and The size of the second target area differs greatly, that is, the number of pixels is unbalanced.
  • the second target area is small compared to the first target area and belongs to a small sample area. Therefore, the cropping range cannot be based on the connected domain range of the first target area, otherwise the proportion of the second target area is too small to be easily lost, and the cropping range cannot be completely based on the connected domain range of the second target area, which can be supplemented
  • the context information is expanded, so it can be expanded by 50%.
  • liver cancer A specific example will be given below to explain the second case in detail, referring to FIG. 6, taking liver cancer as an example.
  • the tissues of the liver image from the outer layer to the inner layer are liver (represented as liver) and liver cancer (represented as tumor).
  • the liver is the first target area
  • the tumor is the second target area.
  • liver and tumor sample imbalance that is, the pixel ratio of the first target area and the second target area is greater than the user tolerance (target value)
  • target value the user tolerance
  • liver belongs to a large sample label
  • tumor belongs to a small sample label. Therefore, when the liver module crops the image, the cropping range is the expansion of the connected domain by 50%. This cropping method can be called a scanning crop.
  • the input in the first module is a full picture.
  • the first module divides liver and tumor, with the focus on liver.
  • the prediction result has two connected domains, where Liver CC1 is larger than the tumor's training set connected domain range, and Liver CC2 is smaller than this range. .
  • Liver CC1 is larger than the tumor's training set connected domain range
  • Liver CC2 is smaller than this range. .
  • Liver CC1 as the center, expand the crop image by 10% according to the statistical range of the liver training set to obtain Liver Crop, which is the image to be cropped after the context information is added on the basis of the first target area, and then the tumor is connected to the domain
  • the range (the range of the connected domain of the second target area) is expanded by 50% from top to bottom, and the input image is scanned from left to right to obtain the input 1, 2, 3 of the Tumor model.
  • Liver CC2 as the center, and expanding the tumor connected area by 50% crop, the fourth input image of the tumor model can be obtained.
  • the initial model in the computer device obtains the segmentation error of each segmentation result based on multiple segmentation results and the label of the sample image.
  • the initial model may use a loss function to obtain the segmentation error of each segmentation result.
  • the type corresponding to the image segmentation module 'S weight is greater than other types of weights.
  • each image segmentation module can focus more on segmenting the corresponding type. For example, for the first image segmentation module, when the segmentation result of the first image segmentation module acquires segmentation error, the weight of the first type is greater than the weights of other types in the loss function. For example, taking four classifications as an example, the first type Has a weight of 0.7, and the weights of the other three categories are all 0.1.
  • the computer device adjusts the model parameters of the initial model based on multiple segmentation errors until it reaches a preset condition and stops to obtain an image segmentation model.
  • the model parameters of the initial model include at least the module parameters of each image segmentation module.
  • the computer device can adjust the model parameters of the initial model based on the multiple segmentation errors, and the model parameters after multiple adjustments can make the initial model segment the sample image again.
  • the error is reduced, that is, the accuracy is higher, so that when the preset conditions are reached, the model is trained.
  • each time the sample image is segmented the segmentation error and the model parameter adjustment process are all iterative processes.
  • the above model training process is a multiple iteration process.
  • the model parameters of the initial model may further include the weight of each image segmentation module, that is, the output of the initial model is an output that integrates the segmentation results of the multiple image segmentation modules, and the output may In order to obtain the weighted summation results of multiple image segmentation modules, that is, after the multiple image segmentation modules obtain multiple segmentation results, the divided images may be output based on the weighted summation of the multiple segmentation results.
  • the above image segmentation modules can be trained separately, and then the weight of each image segmentation module can be trained, or when the module parameters of each image segmentation module are adjusted, the The weight is not limited by the embodiment of the present application.
  • the weights of the multiple image segmentation modules may use Dice values. Evaluation index of segmentation algorithm.
  • the value range of the Dice value may be [0, 1]. The larger the Dice value, the better the segmentation performance.
  • the Dice value may be determined based on cross-validation.
  • the preset condition may be determined based on the gradient descent method. For example, it may be that the segmentation error converges, or that the number of iterations reaches the target number of times.
  • the weights and preset conditions of the above image segmentation module may be determined based on cross-validation.
  • the first iteration stop number may be determined based on the k-fold cross-validation, for example, may be determined based on the five-fold cross-validation. Taking the five-fold cross-validation as an example, the sample image can be divided into five parts, four of which are used as the training set, and the other part is used as the verification set, and then multiple training and verification are performed in another combination.
  • the initial model is trained and verified in different combinations at the same time, so that by training and verifying multiple combinations of sample data, the initial model traverses all sample data, and the trained model is universal Better performance and more accurate segmentation results.
  • the cross-validation process is mainly to verify the trained model through the verification data every time a certain number of iterations are performed. If the segmentation error meets the preset conditions, it can be stopped. If it does not, the above iteration process can be continued Therefore, the embodiments of the present application will not go into details here.
  • the above steps 303 to 306 are the process of training the initial model based on the multiple sample images to obtain an image segmentation model.
  • the initial model can analyze itself based on the sample images to determine the image segmentation module ’s Quantity, so that it can be applied to a variety of scenarios, more versatile, practical and applicable.
  • the computer device After obtaining the image segmentation model from the above training, when the first image to be segmented is acquired, the computer device calls the image segmentation model, and the image segmentation model splits the first image based on multiple image segmentation modules and outputs the second Image, the first image is also the target human tissue image.
  • the input sample image may also be a 3D image
  • the initial model can also convert the 3D image into a 2D image sequence, that is, to obtain a slice of the 3D image, Input the slice into the image segmentation module for segmentation.
  • both the initial model and the image segmentation model include three sub-models of viewing angles.
  • the three sub-models of viewing angles are used to obtain image slices and segment images according to different perspectives, respectively.
  • the three perspective sub-models can respectively obtain image slices according to the X-axis, Y-axis, and Z-axis, and segment them separately, and finally merge the segmentation results of the three perspective sub-models, and output the segmented image.
  • the initial model determines the number of image segmentation modules
  • the number of types of foreground pixel points of the multiple sample images can be used as the number of image segmentation modules in each perspective sub-model. That is, the number of image segmentation modules in each perspective sub-model is the number of types of foreground pixels.
  • the sample image can be sequentially segmented based on multiple image segmentation modules, thereby synthesizing multiple segmentation results of the three perspective sub-models to obtain the final image.
  • the segmentation result of each perspective sub-model among the three perspective sub-models includes image segmentation results of multiple image segmentation modules.
  • each perspective sub-model may also correspond to a weight, which is the above-mentioned Dice value, and the weight of each perspective sub-model may also be determined based on cross-validation.
  • each perspective sub-model includes three image segmentation modules, namely: Model A, Model B, and Model C. The three modules segment the image in sequence to obtain three-stage segmentation results. When the three segmentation results are fused, the segmentation results can be weighted and summed based on the weights (Dice values) of the three modules to obtain a certain Segmentation results of three sub-models of perspective.
  • the weight of each view sub-model may also be considered to obtain the final output image.
  • the computer device may obtain at least one slice of the first image from the three perspective sub-models according to the corresponding perspectives, and each perspective sub Multiple image segmentation modules in the model segment each slice, and output a second image based on the segmentation results of the three perspective sub-models.
  • the segmentation result of each perspective sub-model among the three perspective sub-models includes image segmentation results of multiple image segmentation modules.
  • the process of outputting the second image is the same as during training.
  • the image segmentation results of multiple image segmentation modules of the three perspective sub-models are weighted and summed to output a second image, which is not described in detail in the embodiment of the present application.
  • the embodiment of the present application obtains the image segmentation model by training the initial model based on the sample image, so that when the first image is obtained, the first image can be segmented based on the trained image segmentation model, wherein the initial model can be based on the sample itself
  • the number of pixel types of the image determines the number of image segmentation modules. Therefore, different human tissue images can be directly trained on the basis of this initial model without the need for manual participation to redesign the model. Therefore, the above image segmentation model is common It has good sex, applicability and practicality. Further, the above method is versatile and targeted for all nested clinical organizational structure segmentation, and segmentation performance and timeliness are effectively improved. And the variability of the structure of the initial model makes the method very scalable.
  • FIG. 9 is a flowchart of an image segmentation method provided by an embodiment of the present application.
  • the image segmentation method is applied to a computer device, and the computer device may be the computer device 201 in the foregoing implementation environment.
  • the image segmentation model is mainly called when the first image to be segmented is acquired, and the image segmentation model splits the first image based on a plurality of image segmentation modules, and outputs the second image
  • the image segmentation method may include the following steps:
  • the computer device acquires the first image to be divided.
  • the computer device executes this step 901 when it detects an image segmentation operation, and can also receive the first image to be segmented imported by the user, and can also receive an image segmentation request sent by another computer device, where the image segmentation request carries the first segment to be segmented Image, the first image to be divided is extracted from the image segmentation request, or the image segmentation request may carry relevant information of the first image, and the computer device may perform step 901 based on the relevant information.
  • the computer The device can also acquire the first image to be segmented through the imaging principle.
  • the embodiment of the present application does not limit the specific acquisition method and acquisition timing of the first image to be divided.
  • another computer device may acquire the first image to be divided by imaging principles, and send the first image to be divided to the computer device, and the computer device obtains the first image to be divided, the first image may
  • the following steps may be performed, and the first image is segmented using an image segmentation model obtained by training from the sample image of the target human tissue.
  • the computer device invokes the image segmentation model.
  • the image segmentation model includes multiple image segmentation modules.
  • the number of the plurality of image segmentation modules is determined by the initial model when training the initial model in the embodiment shown in FIG. 3 above. Different image segmentation modules are used to segment different regions of the image.
  • the multiple image segmentation modules can sequentially segment the first image to implement a streaming segmentation scheme.
  • An image segmentation model may be pre-stored in the computer device.
  • the computer device is the computer device shown in FIG. 3, that is, the image segmentation model stored on the computer device is training on the computer device owned.
  • the computer device is not the computer device shown in FIG. 3, but may be an image segmentation model trained on other computer devices, and the computer device may obtain the trained image segmentation model from other computer devices.
  • the image segmentation model can be called from other computer devices in real time. The application examples do not limit this.
  • the image segmentation model may further include three perspective sub-models.
  • the three perspective sub-models are respectively used to obtain image slices and segment the image according to different perspectives.
  • each perspective sub-model includes multiple image segmentation modules.
  • the multiple image segmentation modules respectively correspond to a type of pixel point, that is, the multiple image segmentation modules are respectively used to segment a type of pixel point.
  • the computer device inputs the first image into the image segmentation model, and the image segmentation model splits the first image based on multiple image segmentation modules to obtain multiple segmentation results.
  • step 903 may be: the computer device inputs the first image into the image segmentation model, and three of the image segmentation models Each perspective sub-model obtains at least one slice of the first image according to the corresponding perspective, and each slice is segmented by multiple image segmentation modules in each perspective sub-model, based on the segmentation results of the three perspective sub-models, Output the second image.
  • each image segmentation module can segment and crop the image to obtain a segmentation result.
  • the computer device can be based on The first image segmentation module segments the third image to obtain a first segmentation result.
  • the first image segmentation module is the image segmentation module that is in the order of the two adjacent image segmentation modules.
  • the computer device divides the fourth image cropped based on the first segmentation result based on the second image segmentation module to obtain a second segmentation result, and the second image segmentation module is in the order of the two adjacent image segmentation modules.
  • the fourth image is a partial area of the third image.
  • the fourth image includes pixels of the first type indicated by the first segmentation result, and the first type is a type corresponding to the first image segmentation module.
  • the image segmentation model in the computer device outputs a second image based on multiple segmentation results.
  • the segmentation result of each perspective sub-model in the three perspective sub-models includes image segmentation results of multiple image segmentation modules.
  • the image segmentation model is based on the segmentation results of the three perspective sub-models.
  • the weights corresponding to the three perspective sub-models can also be corresponding to each image segmentation module in each perspective sub-model Weighting, the weighted summation of the image segmentation results of the multiple image segmentation modules of the three viewing angle sub-models to output the second image.
  • the weights corresponding to the three perspective sub-models and the weights corresponding to each image segmentation module in each perspective sub-model are determined based on cross-validation.
  • the computer device can store the second image, of course, the first image and the second image can also be stored correspondingly, if the computer device is an image segmentation request based on other computer devices
  • the above image segmentation process may also send the second image to the other computer device, which is not limited in the embodiment of the present application.
  • the embodiment of the present application obtains the image segmentation model by training the initial model based on the sample image, so that when the first image is obtained, the first image can be segmented based on the trained image segmentation model, wherein the initial model can be based on the sample itself
  • the number of pixel types of the image determines the number of image segmentation modules. Therefore, different human tissue images can be directly trained on the basis of this initial model without the need for manual participation to redesign the model. Therefore, the above image segmentation model is common It has good sex, applicability and practicality.
  • steps in the embodiments of the present application are not necessarily executed in the order indicated by the step numbers. Unless clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. The order is not necessarily sequential, but may be executed in turn or alternately with at least a part of other steps or sub-steps or stages of other steps.
  • a computer device in one embodiment, includes an image segmentation device.
  • the image segmentation device includes various modules, and each module may be implemented in whole or in part by software, hardware, or a combination thereof.
  • FIG. 10 is a schematic structural diagram of an image segmentation device according to an embodiment of the present application.
  • the device includes:
  • the obtaining module 1001 is used to obtain multiple sample images
  • the training module 1002 is used to call the initial model, input the multiple sample images into the initial model, and train the initial model based on the multiple sample images to obtain an image segmentation model.
  • the initial model is used to determine the multiple samples
  • the number of pixel types of an image determines the number of image segmentation modules, and different image segmentation modules are used to segment different regions of the image;
  • the segmentation module 1003 is used to call the image segmentation model when the first image to be segmented is acquired, and the image segmentation model splits the first image based on a plurality of image segmentation modules, and outputs a second image.
  • Each sample image and the first image are target human tissue images.
  • the training module 1002 is configured to use the number of types of foreground pixels of multiple sample images as the number of image segmentation modules in the initial model.
  • each sample image carries a label, which is used to indicate the target segmentation result of the sample image
  • the training module 1002 is also used to analyze the labels of the multiple sample images to obtain the number of types of pixels of the multiple sample images.
  • the segmentation module 1003 is used to:
  • the third image is segmented based on the first image segmentation module to obtain a first segmentation result, and the first image segmentation module is the two adjacent image The first image segmentation module in the segmentation module;
  • the fourth image cropped based on the first segmentation result is segmented to obtain a second segmentation result
  • the second image segmentation module is an image in the order of the two adjacent image segmentation modules In the segmentation module, the fourth image is a partial area of the third image.
  • the plurality of image segmentation modules respectively correspond to a type of pixel points; the fourth image includes pixels of the first type indicated by the first segmentation result, and the first type is the first image The type corresponding to the segmentation module; when the segmentation error of the segmentation result of each image segmentation module is obtained during the training of the initial model, the weight of the type corresponding to the image segmentation module is greater than the weights of other types.
  • the acquisition module 1001 is also used to acquire the pixel ratio between adjacent target regions in multiple target regions corresponding to the multiple sample images during the training of the initial model, the target region is The area where the pixels of the target type in the multiple sample images are located;
  • the device also includes:
  • the cropping module is used to crop the image according to the size relationship between the pixel ratio between the adjacent target areas in the multiple target areas and the target value when each of the multiple image splitting modules crops the image. To crop.
  • the acquisition module 1001 is also used to acquire the connected domain range of multiple target regions corresponding to the multiple sample images during the training process of the initial model;
  • the cropping module is used to crop the image according to the size relationship between the pixel ratio between adjacent target areas in the multiple target areas and the target value when each of the multiple image segmentation modules cuts the image, and the The range of connected domains of multiple target areas is used to crop the image.
  • the cropping module is used to:
  • the third image For cropping the first target area is the area of the first type of pixel corresponding to the first image segmentation module, and the second target area is the area of the second type of pixel corresponding to the second image segmentation module.
  • the cropping module is also used to:
  • the third image is cropped based on the connected domain range of the first target area
  • the third image is cropped based on the connected domain range of the second target area.
  • the size of the cropped fourth image is obtained based on the connected domain range and the first coefficient
  • the size of the cropped fourth image is obtained based on the connected domain range of the second target area and the second coefficient, the first coefficient is less than The second coefficient.
  • both the initial model and the image segmentation model include three perspective sub-models, and the three perspective sub-models are respectively used to obtain image slices and segment the image according to different perspectives;
  • the training module 1002 is used for the number of types of foreground pixels of the multiple sample images as the number of image segmentation modules in each perspective sub-model;
  • the segmentation module 1003 is used to obtain at least one slice of the first image from the three perspective sub-models according to corresponding perspectives, and each slice is segmented by multiple image segmentation modules in each perspective sub-model Based on the segmentation results of the three perspective sub-models, a second image is output.
  • the segmentation results of each of the three perspective sub-models include image segmentation results of multiple image segmentation modules
  • the segmentation module 1003 is also used to determine the multiple image segmentation
  • the image segmentation results are weighted and summed to output the second image.
  • the weights corresponding to the three perspective sub-models and the weights corresponding to each image segmentation module in each perspective sub-model are determined based on cross-validation.
  • the training module 1002 is also used to acquire the image segmentation sub-module corresponding to the number of images for each image segmentation module according to the number of images of the multiple sample images during the training of the initial model
  • the image segmentation module includes at least one image segmentation submodule, and different image segmentation submodules have different depths.
  • the device provided by the embodiment of the present application obtains the image segmentation model by training the initial model based on the sample image, so that when the first image is acquired, the first image can be segmented based on the trained image segmentation model, where the initial model
  • the number of image segmentation modules can be determined based on the number of pixel types of the sample image. Therefore, different human tissue images can be directly trained on the basis of this initial model without human participation, and the model is redesigned. Therefore, the above image
  • the segmentation model has good versatility, applicability and practicality.
  • the image segmentation device provided in the above embodiments only uses the above-mentioned division of each functional module as an example when dividing an image.
  • the above-mentioned functions can be allocated by different functional modules according to needs.
  • the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.
  • the image segmentation apparatus and the image segmentation method embodiment provided in the above embodiments belong to the same concept. For the specific implementation process, refer to the method embodiment, and details are not described here.
  • the above-mentioned computer device may be provided as a terminal shown in FIG. 11 described below, or may be provided as a server shown in FIG. 12 described below, which is not limited in the embodiment of the present application.
  • the terminal 1100 may be: a smartphone, a tablet computer, an MP3 player (Moving Pictures Experts Group Audio Audio Layer III, motion picture expert compression standard audio level 3), MP4 (Moving Pictures Experts Group Audio Audio Layer IV, motion picture expert compression standard audio Level 4) Player, laptop or desktop computer.
  • the terminal 1100 may also be called other names such as user equipment, portable terminal, laptop terminal, and desktop terminal.
  • the terminal 1100 includes a processor 1101 and a memory 1102.
  • the processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on.
  • the processor 1101 can adopt at least one hardware form from DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). achieve.
  • the processor 1101 may also include a main processor and a coprocessor.
  • the main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor for processing data in the standby state.
  • the processor 1101 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used to render and draw content that needs to be displayed on the display screen.
  • the processor 1101 may further include an AI (Artificial Intelligence, Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, Artificial Intelligence
  • the memory 1102 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 1102 may further include a high-speed random access memory, and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction that is executed by the processor 1101 to implement the image segmentation provided by the method embodiment in the present application Model training method or image segmentation method.
  • the terminal 1100 may optionally include a peripheral device interface 1103 and at least one peripheral device.
  • the processor 1101, the memory 1102 and the peripheral device interface 1103 may be connected by a bus or a signal line.
  • Each peripheral device may be connected to the peripheral device interface 1103 through a bus, a signal line, or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 1104, a touch display screen 1105, a camera 1106, an audio circuit 1107, a positioning component 1108, and a power supply 1109.
  • the peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102.
  • the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1101, the memory 1102, and the peripheral device interface 1103 or Both can be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 1104 communicates with a communication network and other communication devices through electromagnetic signals.
  • the radio frequency circuit 1104 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal.
  • the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on.
  • the radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity, wireless fidelity) networks.
  • the radio frequency circuit 1104 may further include a circuit related to NFC (Near Field Communication), which is not limited in this application.
  • the display screen 1105 is used to display a UI (User Interface).
  • the UI may include graphics, text, icons, video, and any combination thereof.
  • the display screen 1105 also has the ability to collect touch signals on or above the surface of the display screen 1105.
  • the touch signal can be input to the processor 1101 as a control signal for processing.
  • the display screen 1105 can also be used to provide virtual buttons and / or virtual keyboards, also called soft buttons and / or soft keyboards.
  • the display screen 1105 there may be one display screen 1105, which is provided with the front panel of the terminal 1100; in other embodiments, there may be at least two display screens 1105, which are respectively provided on different surfaces of the terminal 1100 or have a folded design; In some embodiments, the display screen 1105 may be a flexible display screen, which is disposed on the curved surface or folding surface of the terminal 1100. Even, the display screen 1105 can also be set as a non-rectangular irregular figure, that is, a shaped screen.
  • the display screen 1105 may be made of LCD (Liquid Crystal), Liquid Crystal Display (OLED), Organic Light-Emitting Diode (Organic Light Emitting Diode) and other materials.
  • the camera assembly 1106 is used to collect images or videos.
  • the camera assembly 1106 includes a front camera and a rear camera.
  • the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal.
  • there are at least two rear cameras which are respectively one of the main camera, the depth-of-field camera, the wide-angle camera, and the telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera Integrate with wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting function or other fusion shooting functions.
  • the camera assembly 1106 may also include a flash.
  • the flash can be a single-color flash or a dual-color flash. Dual color temperature flash refers to the combination of warm flash and cold flash, which can be used for light compensation at different color temperatures.
  • the audio circuit 1107 may include a microphone and a speaker.
  • the microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1101 for processing, or input them to the radio frequency circuit 1104 to realize voice communication.
  • the microphone can also be an array microphone or an omnidirectional acquisition microphone.
  • the speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves.
  • the speaker can be a traditional thin-film speaker or a piezoelectric ceramic speaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible by humans, but also convert electrical signals into sound waves inaudible to humans for ranging and other purposes.
  • the audio circuit 1107 may further include a headphone jack.
  • the positioning component 1108 is used to locate the current geographic location of the terminal 1100 to implement navigation or LBS (Location Based Service, location-based service).
  • the positioning component 1108 may be a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China, the Grenas system of Russia, or the Galileo system of the European Union.
  • the power supply 1109 is used to supply power to various components in the terminal 1100.
  • the power supply 1109 may be alternating current, direct current, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal 1100 further includes one or more sensors 1110.
  • the one or more sensors 1110 include, but are not limited to: an acceleration sensor 1111, a gyro sensor 1112, a pressure sensor 1113, a fingerprint sensor 1114, an optical sensor 1115, and a proximity sensor 1116.
  • the acceleration sensor 1111 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established with the terminal 1100.
  • the acceleration sensor 1111 can be used to detect the components of gravity acceleration on three coordinate axes.
  • the processor 1101 may control the touch display screen 1105 to display the user interface in a landscape view or a portrait view according to the gravity acceleration signal collected by the acceleration sensor 1111.
  • the acceleration sensor 1111 can also be used for game or user movement data collection.
  • the gyro sensor 1112 can detect the body direction and rotation angle of the terminal 1100, and the gyro sensor 1112 can cooperate with the acceleration sensor 1111 to collect a 3D action of the user on the terminal 1100. Based on the data collected by the gyro sensor 1112, the processor 1101 can realize the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 1113 may be disposed on the side frame of the terminal 1100 and / or the lower layer of the touch display 1105.
  • the pressure sensor 1113 can detect the user's grip signal on the terminal 1100, and the processor 1101 can perform left-right hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 1113.
  • the processor 1101 controls the operability control on the UI interface according to the user's pressure operation on the touch display screen 1105.
  • the operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 1114 is used to collect a user's fingerprint, and the processor 1101 recognizes the user's identity according to the fingerprint collected by the fingerprint sensor 1114, or the fingerprint sensor 1114 recognizes the user's identity according to the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 1101 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 1114 may be provided on the front, back, or side of the terminal 1100. When a physical button or manufacturer logo is provided on the terminal 1100, the fingerprint sensor 1114 may be integrated with the physical button or manufacturer logo.
  • the optical sensor 1115 is used to collect the ambient light intensity.
  • the processor 1101 may control the display brightness of the touch display 1105 according to the ambient light intensity collected by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the touch display 1105 is increased; when the ambient light intensity is low, the display brightness of the touch display 1105 is decreased.
  • the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 according to the ambient light intensity collected by the optical sensor 1115.
  • the proximity sensor 1116 also called a distance sensor, is usually provided on the front panel of the terminal 1100.
  • the proximity sensor 1116 is used to collect the distance between the user and the front of the terminal 1100.
  • the processor 1101 controls the touch display 1105 to switch from the bright screen state to the breathing state;
  • the processor 1101 controls the touch display screen 1105 to switch from the screen-on state to the screen-on state.
  • FIG. 11 does not constitute a limitation on the terminal 1100, and may include more or fewer components than those illustrated, or combine certain components, or adopt different component arrangements.
  • FIG. 12 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 1200 may have a relatively large difference due to different configurations or performances, and may include one or more than one central processing unit (CPU) 1201 and one Or more than one memory 1202, wherein at least one instruction is stored in the memory 1202, and the at least one instruction is loaded and executed by the processor 1201 to implement the image segmentation model training method or image provided by the foregoing method embodiments Segmentation method.
  • the server may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output.
  • the server may also include other components for implementing device functions, which will not be repeated here.
  • the above-mentioned computer device may be provided as a server shown in FIG. 13 described below.
  • the server includes a processor, memory, network interface, and database connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, a computer program, and a database.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer device is used to store image data.
  • the network interface of the computer device is used to communicate with external terminals through a network connection.
  • the computer program is executed by the processor to implement an image segmentation method or image segmentation model training method.
  • the above-mentioned computer device may be provided as a terminal shown in FIG. 14 described below.
  • the terminal includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and computer programs.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the network interface of the computer device is used to communicate with external terminals through a network connection.
  • the computer program is executed by the processor to implement an image segmentation method or image segmentation model training method.
  • the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or may be a button, a trackball or a touchpad provided on the computer device housing , Can also be an external keyboard, touchpad or mouse.
  • FIGS. 13 and 14 are only block diagrams of some structures related to the solution of the present application, and do not constitute a limitation on the server and terminal to which the solution of the present application is applied.
  • the terminal may include more or less components than shown in the figure, or combine certain components, or have different component arrangements.
  • the image segmentation device provided by the present application may be implemented in the form of a computer-readable instruction, and the computer-readable instruction may run on a server as shown in FIG. 13 or may be as shown in FIG. Run on the terminal.
  • the memory of the server or the terminal may store various program modules constituting the image segmentation device, such as the acquisition module 1001, the training module 1002, and the segmentation module 1003.
  • the computer-readable instructions formed by the various program modules cause the processor to execute the steps in the image segmentation method or the image segmentation model training method of the various embodiments of the present application described in this specification.
  • An embodiment of the present application provides a computer-readable storage medium that stores computer-readable instructions, which are loaded by a processor and have an image segmentation method or image segmentation model to implement the above-described embodiments Operations in training methods.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous chain (Synchlink) DRAM
  • SLDRAM synchronous chain (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种图像分割方法、装置、计算机设备及存储介质，属于计算机技术领域。方法包括：获取多个样本图像；调用初始模型，将多个样本图像输入初始模型中，基于多个样本图像，对初始模型进行训练，得到图像分割模型，初始模型用于根据多个样本图像的像素点的类型数量确定图像分割模块的数量；当获取到待分割的第一图像时，调用图像分割模型，由图像分割模型基于多个图像分割模块，对第一图像进行分割，输出第二图像。

Description

图像分割方法、装置、计算机设备及存储介质
本申请要求于2018年10月16日提交中国专利局，申请号为201811204371.6，申请名称为“图像分割方法、装置、计算机设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种图像分割方法、装置、计算机设备及存储介质。
背景技术
随着计算机技术的发展，图像分割技术应用越来越广泛，例如，医学图像分割、自然图像分割等。其中，图像分割技术是指把图像分成若干个特定的、具有独特性质的区域并提出感兴趣目标的技术。人体组织通常有着固定的范围限制以及形态分布特征，根据不同组织分布情况设计特定的图像分割模型能有效地提高分割的准确性。其中，人体组织的一种分布情况为嵌套式，也就是，人体的一个组织内部有另一个组织。例如，脑部肿瘤即为一种嵌套式组织，从嵌套的外层到内层依次为水肿、非增强型肿瘤、增强型肿瘤。
目前，如图1所示，图像分割方法通常采用级联各向异性卷积神经网络(Cascaded Anisotropic Convolutional Neural Networks，Cascaded ACNN)，该网络用于对脑部肿瘤进行分割。由于脑部肿瘤需要分割出上述三种组织，该网络包括三个不同类型的分割模型：WNET(Whole Tumor Network)、TNET(Tumor Network)和ENET(Enhancing Tumor Network)，该三个分割模型分别用于对图像进行不同类型的分割，也即是分别用于分割出上述三种组织。图像分割方法通常是将图像输入该网络后，WNET对完整的图像进行分割，得到该图像的水肿区域，将水肿区域的图像输入TNET中，TNET对水肿区域的图像进行分割，得到该图像的非增强型肿瘤所在区域，将非增强型肿瘤所在区域输入ENET中，ENET对非增强型肿瘤所在区域进行分割，得到增强型肿瘤所在区域，最终Cascaded ACNN将三个模型得到的区域按照从小到大的顺序重叠在一起，得到分割后的图像。
上述Cascaded ACNN仅适用于对脑部肿瘤进行分割，如果需要对其他嵌套式人体组织图像进行分割，则需要技术人员基于其他嵌套式人体组织图像的分布情况，重新设计模型，而无法该网络的基础上直接进行训练，因此，上述图像分割方法的通用性、适用性和实用性差。
发明内容
本申请提供的各种实施例，提供了一种图像分割方法、装置、计算机设备及存储介质。所述技术方案如下：
一方面，提供了一种图像分割方法，所述方法包括：
计算机设备获取多个样本图像；
所述计算机设备调用初始模型，将所述多个样本图像输入初始模型中，基于所述多个样本图像，对所述初始模型进行训练，得到图像分割模型，所述初始模型用于根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，不同的图像分割模块用于对图像的不同区域进行分割；
当获取到待分割的第一图像时，所述计算机设备调用所述图像分割模型，由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，输出第二图像，所述多个样本图像和所述第一图像均为目标人体组织图像。
一方面，提供了一种图像分割装置，所述装置包括：
获取模块，用于获取多个样本图像；
训练模块，用于调用初始模型，将所述多个样本图像输入初始模型中，基于所述多个样本图像，对所述初始模型进行训练，得到图像分割模型，所述初始模型用于根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，不同的图像分割模块用于对图像的不同区域进行分割；
分割模块，用于当获取到待分割的第一图像时，调用所述图像分割模型，由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，输出第二图像，所述多个样本图像和所述第一图像均为目标人体组织图像。
一方面，提供了一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行上述实施例所述的方法。
一方面，提供了一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述实施例所述的方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请背景技术提供的一种Cascaded ACNN的分割流程示意图；
图2是本申请实施例提供的一种图像分割方法的实施环境；
图3是本申请实施例提供的一种图像分割模型训练方法的流程图；
图4是本申请实施例提供的一种流式分割流程的示意图；
图5是本申请实施例提供的一种图像裁剪方式的示意图；
图6是本申请实施例提供的一种图像裁剪方式的示意图；
图7是本申请实施例提供的一种图像分割模型的结构示意图；
图8是本申请实施例提供的一种分割结果融合方式的示意图；
图9是本申请实施例提供的一种图像分割方法的流程图；
图10是本申请实施例提供的一种图像分割装置的结构示意图；
图11是本申请实施例提供的一种终端的结构示意图；
图12是本申请实施例提供的一种服务器的结构示意图；
图13是本申请实施例提供的一种服务器的结构示意图；
图14是本申请实施例提供的一种终端的结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。基于本申请中的实施例，本领域普 通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图2是本申请实施例提供的一种图像分割方法的实施环境，参见图2，该实施环境中可以包括多个计算机设备。其中，该多个计算机设备可以通过有线连接方式实现数据交互，也可以通过无线网络连接方式实现数据交互，本申请实施例对此不作限定。
在本申请实施例中，计算机设备201可以用于对图像进行分割，在一些实施例中，该图像可以是医学图像，也就是人体组织图像，该人体组织的分布情况为嵌套式，也就是该人体组织图像中一个组织内部有另一个组织。该图像分割方法可以适用于对任何嵌套式的组织的分析场景，例如，肝与肝癌分析、胰腺与胰腺癌分析、肺与肺癌分析、脑部肿瘤分析或脑癌分析等场景。当然，该图像分割方法也可以适用于其它人体组织图像分割场景，本申请实施例在此不一一列举。当然，该图像也可以是其它类型的图像，则该图像分割方法也可以应用于其他图像分割场景中，例如，风景图像分割场景。
该计算机设备202可以用于采集图像，并将采集到的图像发送至计算机设备201，由计算机设备201提供图像分割服务。在一些实施例中，该计算机设备201也可以采集图像，并对采集到的图像进行分割，本申请实施例对此不作限定。在一些实施例中，该计算机设备202也可以用于存储从其他计算机设备处获取到的图像，该计算机设备201可以从该计算机设备202中获取到存储的图像进行分割。
具体地，该计算机设备201和计算机设备202均可以被提供为终端，也可以被提供为服务器，本申请实施例对此不作限定。
图3是本申请实施例提供的一种图像分割模型训练方法的流程图，该图像分割模型训练方法可以应用于计算机设备中，该计算机设备可以为上述实施环境中的计算机设备201，也可以是其他计算机设备。也就是，可以在上述计算机设备201上训练得到图像分割模型，也可以在其他计算机设备上训练得到图像分割模型后，将该图像分割模型处理为配置文件，将该配置文件发送至计算机设备201，则该计算机设备201中就存储有图像分割模型。当然，也可以由计算机设备201在有图像分割需求时，调用其它计算机设备上训练得到的图像分割模型，本申请实施例对此不作限定。参见图3，该方法可以包括以下步骤：
301、计算机设备获取多个样本图像，每个样本图像携带有标签，该标签用于指示样本图像的目标分割结果。
该多个样本图像为目标人体组织图像，计算机设备可以基于多个样本图像对初始模型进行训练，得到图像分割模型，这样得到的图像分割模型可以对目标人体组织图像进行分割。其中，该目标人体组织图像可以为上述嵌套式组织的图像，例如，肝部图像、胰腺图像、肺部图像、脑部图像等。
在一些实施例中，在该计算机设备中可以存储有该多个样本图像，在需要进行图像分割模型训练时，该计算机设备就可获取到该存储的多个样本图像。需要说明的是，每个样本图像还可以携带用于指示目标分割结果的标签，其中，该目标分割结果是指该样本图像的正确的分割结果，或是指该样本图像的真实的分割结果。这样在模型训练过程中，可以获知训练中的模型对样本图像的分割是否准确，可以获知是否需要继续对模型进行训练，从而训练得到的模型对样本图像进行分割时可以得到该目标分割结果，或者非常接近该目标分割结果。
在一些实施例中，该多个样本图像也可以存储于其他计算机设备，该计算机设备在需要进行图像分割模型训练时，可以从其他计算机设备处获取，本申请实施例对此不作限定。例如，该多个样本图像可以存储于图像数据库中，每个样本图像还携带有标签。则该步骤301就可以为计算机设备从图像数据库中获取多个样本图像。
302、计算机设备调用初始模型，将该多个样本图像输入该初始模型中。
计算机设备获取多个样本图像后，可以调用初始模型，基于该多个样本图像对初始模型进行训练，得到图像分割模型，以便于后续能够基于该图像分割模型对获取到的第一图像进行准确分割。
在一些实施例中，在计算机设备中可以存储有初始模型，并在该步骤302中直接调用该存储的初始模型。在一些实施例中，在该计算机设备中也可以不存储初始模型，在该计算机设备需要训练模型时，从其他计算机设备处调用初始模型，并执行后续模型训练过程，本申请实施例对具体采用哪种实现方式不作限定。
需要说明的是，该初始模型的模型参数为初始值，计算机设备可以将该多个样本图像作为训练样本和验证样本，对初始模型进行训练，也就是通过样本图像对初始模型的模型参数进行调整，以使得多次调整后的模型参数能够在对第一图像进行分割时，得到的分割结果更准确。
具体地，计算机设备将该多个样本图像输入初始模型中，可以由初始模型对每个样本图像进行分割，基于初始模型的分割结果和样本图像的标签，也就是样本图像的目标分割结果，确定初始模型的分割能力，从而可以通过调整该初始模型的模型参数，以不断提高初始模型的分割能力，以使得后续训练得到的图像分割模型能够准确分割。
在一些实施例中，对于该初始模型的模型参数，也即是上述初始值，可以基于多种样本图像预训练得到，该多种样本图像可以包括多种人体组织图像，该多种人体组织均为上述嵌套式组织，这样该初始模型通过预训练过程获取到了先验知识，使得该初始模型已经对嵌套式组织的图像进行了一定的了解，可以用于模拟医学生在各个科室轮转，从而该医学生可以具备一定的医学知识或临床知识。这样得到的初始模型在后续进行模型训练时，可以有效减少模型训练次数，也可以基于各种嵌套式组织图像将该初始模型，应用于对各种嵌套式组织图像进行分割，可以有效提高该初始模型以及基于该初始模型训练得到的图像分割模型的实用性和通用性。
在一些实施例中，该计算机设备可以基于获取到的多个样本图像和历史处理数据中的样本图像进行匹配，获取该多个样本图像和历史样本图像的相似度，然后将相似度最大的样本图像的图像分割模型的模型参数作为上述初始模型的模型参数，也就是上述初始值，这样考虑到历史处理数据中可能处理过类似的样本图像，考虑到样本图像相似，对样本图像进行分割时所需的参数则可能相似，通过直接获取历史处理数据中的模型参数作为初始模型的模型参数，可以有效减少模型训练过程中的迭代次数，减少大量计算量，提高了图像分割模型训练过程的效率。
上述仅提供了两种初始模型的模型参数的设置示例，当然，该初始模型的模型参数也可以由相关技术人员预先设置，本申请实施例对此不作限定。
303、计算机设备中的初始模型根据该多个样本图像的像素点的类型数量，确定图像分割模块的数量。
其中，不同的图像分割模块用于对图像的不同区域进行分割。在本申请实施例中，初始模型可以采用流式分割方案，也就是通过多个图像分割模块依次对样本图像进行分割，对于相邻的两个图像分割模块，前面的图像分割模块对图像进行分割后，可以对原图像进行裁剪，并将裁剪后的图像输入后面的图像分割模块，从而后面的图像分割模块可以基于前一个图像分割模块的分割结果 继续进行分割。这样递进式对图像进行多次分割，且关注点一步一步集中，实现了粗糙和细致结合的分割方式，使得分割结果更准确，也适用于不同难度的分割任务。例如，如图4所示，可以先进行一步分割，再对分割得到的区域进行更细致地分割。
该初始模型可以支持各种嵌套式组织的图像的分割需求，且可以支持多类型分割。由于不同人体组织图像，分割过程中需要确定像素点的类型数量不同，例如，脑部肿瘤分割场景需要确定像素点的类型可以包括4种：背景、水肿、非增强型肿瘤和增强型肿瘤。而肝癌分割场景需要确定像素点的类型可以包括三种：背景、肝和肝癌。该初始模型可以基于该类型数据，自行确定图像分割模块的数量，因而，不同的人体组织图像均可以采用该初始模型进行训练从而满足分割需求，有效提高了初始模型的通用性和实用性。
具体地，该步骤303可以为：初始模型将该多个样本图像的前景像素点的类型数量作为该初始模型中的图像分割模块的数量。像素点的类型可以包括至少两个，该至少两个类型可以包括两类：一类为背景，另一类为前景，前景即为该至少两个类型中的背景之外的其它一种或多种类型。相应地，对应的像素点分别为背景像素点和前景像素点。也就是，类型为背景的像素点为背景像素点，类型为前景的像素点为前景像素点。初始模型确定了多个图像分割模块，该多个图像分割模块分别对应像素点的一种类型，也就是每个图像分割模块会着重分割像素点的一种类型。
例如，如果该人体组织图像为脑部图像，前景像素点的类型数量为3，也就是，前景像素点有三种类型：水肿、非增强型肿瘤和增强型肿瘤，相应地，初始模型即可确定图像分割模块的数量为3。这样通过该三个图像分割模块依次对图像进行分割，第一个图像分割模块着重分割出水肿区域，第二图像分割模块着重分割出非增强型肿瘤区域，而第三个图像分割模块着重分割出增强型肿瘤区域，从而得到图像的分割结果。
需要说明的是，该初始模型可以直接获取该样本图像的前景像素点的类型数量，也可以获取像素点的类型数量，并将该像素点的类型数量减一，得到前景像素点的类型数量，本申请实施例对具体的实现方式不作限定。
在一些实施例中，该多个样本图像的标签中的目标分割结果用于指示样本图像的每个像素点的类型，则该多个样本图像的像素点的类型数据可以基于多个样本图像的标签得到。相应地，该步骤303之前，该计算机设备的初始模型 还可以对该多个样本图像的标签进行分析，得到该多个样本图像的像素点的类型数量，从而可以执行该步骤303，基于类型数量，确定图像分割模块的数量。例如，初始模型可以统计标签中的目标分割结果中像素点的类型数量，也可以仅统计前景像素点的类型数量。
相关技术中，通常需要由技术人员根据自身经验或对需要进行分割的人体组织图像进行分析，人工确定初始模型中的图像分割模块的数量，且图像分割模块的数量固定，在需要对其它人体组织图像进行分割时，需要技术人员重新确定图像分割模块的数量，而不能直接在初始模型的基础上进行训练，因此，初始模型不具有通用性。而本申请实施例中的初始模型可以自行对样本图像进行分析，确定图像分割模块的数量，该图像分割模块的数量可变，在需要对其它人体组织图像进行分割时，直接获取其它人体组织图像对初始模型进行训练即可，该初始模型可以自行确定图像分割模块的数量，因此，本申请实施例提供的初始模型可以适用于对多种人体组织图像进行分割的场景，具备通用性，实用性更好。
304、初始模型中的多个图像分割模块依次对每个样本图像进行分割，得到每个图像分割模块对样本图像的分割结果。
在确定图像分割模块的数量后，初始模型即可基于多个图像分割模块依次对样本图像进行分割，得到分割结果。具体地，对于每个图像分割模块，可以基于模块参数，对输入的图像进行特征提取，得到图像的特征。然后可以基于提取的特征，对图像中每个像素点进行分类，得到分割结果。
在一些实施例中，该图像分割模块中可以包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。例如，该图像分割模块可以采用深度残差网络(Deep residual network，ResNet)实现。该图像分割模块可以包括两个图像分割子模块：ResNet-18和ResNet-152，其中，ResNet-152的深度大于ResNet-18的深度。在该初始模型的训练过程中，对于每个图像分割模块，初始模型还可以根据该多个样本图像的图像数量，获取该图像数量对应的图像分割子模块作为该图像分割模块。这样可以针对样本图像的图像数量，选择合适的图像分割子模块来进行训练，从而可以避免出现过拟合(over-fitting)现象或训练后的模型的分割能力差的问题。在此仅以每个图像分割子模块采用二维(Two-dimensional，2D)网络为基础网络为例进行说明，具体地，每个图像分割子模块也可以使用三维Three-dimensional，3D)网络作为基础网络，本申请 实施例对此不作限定。
具体地，该初始模型中还可以存储有图像数量与图像分割子模块的对应关系，初始模型基于该对应关系，进行上述图像分割子模块的选择步骤。在一些实施例中，图像数量越大，获取的图像分割子模块的深度越大。这样可以有效应对小数据的情况，在样本数量很少时也可以训练模型，得到分割效果较好的图像分割模型。
进一步地，以该图像分割模块包括两个图像子模块为例，该图像分割模块的获取步骤可以为：当该多个样本图像的图像数量大于预设数量时，初始模型获取第一图像分割子模块；当该多个样本图像的图像数量小于或等于预设数量时，初始模型获取第二图像分割子模块。其中，该第一图像分割子模块的深度大于第二图像分割子模块的深度。预设数量可以由相关技术人员预先设置，本申请实施例对该预设数量的具体取值不作限定。
例如，第一图像分割子模块可以为ResNet-152，第二图像分割子模块可以为ResNet-18，以该预设数量为100为例，上述图像分割模块的获取步骤可以为：当样本图像的图像数量小于或等于100时，可以采用ResNet-18作为基础模型，当样本图像的图像数量大于100时，可以采用ResNet-101作为基础模型。
在一些实施例中，每个图像分割模块均可以基于Unity Networking(unet)进行改进，unet因其独特的渐进式上采样以及跳跃式连接(skip connection)结构，特别适用于人体组织图像中细微结构的分割。
初始模型确定了每个图像分割模块的基础模型(图像分割子模块)后，则可以基于该多个图像分割模块依次对样本图像进行分割。具体地，对于该多个图像分割模块中相邻的两个图像分割模块，初始模型可以基于第一图像分割模块对第三图像进行分割，得到第一分割结果，该第一图像分割模块为该相邻的两个图像分割模块中顺序在前的图像分割模块。初始模型再基于第二图像分割模块对基于该第一分割结果裁剪得到的第四图像进行分割，得到第二分割结果，该第二图像分割模块为该相邻的两个图像分割模块中顺序在后的图像分割模块，该第四图像为该第三图像的部分区域。
其中，该第一分割结果与该第二分割结果用于指示图像的每个像素点为至少两个类型中每个类型的概率。第三图像是指输入该第一图像分割模块的图像，第四图像为第一图像分割模块基于第一分割结果对第三图像进行裁剪得到的图像。该第四图像包括该第一分割结果所指示的第一类型的像素点，该第一类型 为该第一图像分割模块对应的类型。
同理地，第二图像分割模块对应的类型为第二类型。由于目标人体组织为嵌套式组织，第二类型的像素点所在区域在第一类型的像素点所在区域的内部。可以先着重分割第一类型的像素点，再在第一类型的像素点所在区域内更细致地着重分割第二类型的像素点。需要说明的是，每个图像分割模块均可以对像素点进行分类，确定该像素点为各个类型的概率，而不是仅针对模块对应的类型，而只是更着重分割模块对应的类型。
也即是，第一图像分割模块对第三图像进行分割，确定了各个像素点为各个类型的概率，并初步以该概率确定各个像素点的类型，第一图像分割模块着重分割第一类型，所以可以将包括第一类型的像素点所在区域的第四图像输入第二图像分割模块，第二图像分割模块则可以继续对第四图像进行分割，更着重分割第二类型。如果上述第一图像分割模块为初始模型的多个图像分割模块中的第一个图像分割模块，则上述第三图像即为输入的样本图像本身。
在一些实施例中，考虑到如果样本图像的多个目标区域之间的像素点相差很大，在采样过程中可能会导致某个目标区域的像素点很少甚至消失，使得分割结果不准确，则在上述裁剪过程中还可以设置有：样本图像的多个目标区域之间的像素点比值不同时，对样本图像的裁剪方式还可以不同。具体地，上述步骤303中，在该初始模型的训练过程中，初始模型还可以获取该多个样本图像对应的多个目标区域中相邻目标区域之间的像素点比值，该目标区域为该多个样本图像中目标类型的像素点所在区域。相应地，在该步骤304中，在该多个图像分割模块中每个图像分割模块对图像进行裁剪时，图像分割模块可以根据该多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，对图像进行裁剪。
其中，该目标数值是指像素点比值的阈值，可以用来衡量目标区域之间的像素点比值是否超出用户容忍度，也即是用于衡量多个目标区域之间比例是否失衡，该目标数值可以由相关技术人员预先设置，也可以由计算机设备为用户提供设置选项或输入功能，获取用户设置的数值作为该目标数值，或对用户设置的数值进行处理得到，例如，用户可以输入3，计算机设备可以对3取倒数，得到目标数值1/3，当然，也可以将3作为目标数值，本申请实施例对该目标数值的具体取值不作限定。
在一些实施例中，上述裁剪过程中图像分割模块对图像进行裁剪时，可以 基于该图像分割模块对应类型的目标区域的尺寸，确定裁剪范围，例如，可以将当前模块对应类型的目标区域作为中心，并外扩一定的百分比(例如10％)进行裁剪，这样输入下一个模块的图像不仅包括当前模块确定出的目标区域，还包括目标区域周围的一些像素信息，可以使得下一个模块也可以对这部分区域再次分割，以避免由于某个模块的分割不准确导致的误差，提高了图像分割的准确性。
具体地，在步骤303中，在该初始模型的训练过程中，初始模型还可以获取该多个样本图像对应的多个目标区域的连通域范围。该步骤可以通过对样本图像的标签进行连通域处理得到，这样通过对样本图像进行统计处理，得到一个标准值，以该标准值来确定合适的裁剪范围。相应地，该步骤304中，在该多个图像分割模块中每个图像分割模块对图像进行裁剪时，图像分割模块可以根据该多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，以及该多个目标区域的连通域范围，对图像进行裁剪。
例如，以上述相邻的两个图像分割模块来说明，该步骤304可以为：对于第一图像分割模块，第一图像分割模块根据第一目标区域与第二目标区域的像素点比值与目标数值的大小关系，以及该第一目标区域或第二目标区域的连通域范围，对该第三图像进行裁剪，该第一目标区域为该第一图像分割模块对应的第一类型的像素点所在区域，该第二目标区域为该第二图像分割模块对应的第二类型的像素点所在区域。
上述过程中根据第一目标区域还是第二目标区域的连通域范围的选择可以根据上述大小范围确定。具体地，可以包括两种选择情况：情况一、当该第一目标区域与第二目标区域的像素点比值小于目标数值时，第一图像分割模块基于该第一目标区域的连通域范围，对该第三图像进行裁剪。情况二、当该第一目标区域与第二目标区域的像素点比值大于或等于目标数值时，第一图像分割模块基于该第二目标区域的连通域范围，对该第三图像进行裁剪。
进一步地，在情况一中，当该第一目标区域与第二目标区域的像素点比值小于目标数值时，裁剪后的第四图像的尺寸基于该第一目标区域的连通域范围和第一系数得到。在情况二中，当该第一目标区域与第二目标区域的像素点比值大于或等于目标数值时，裁剪后的第四图像的尺寸基于该第二目标区域的连通域范围和第二系数得到，该第一系数小于该第二系数，该第一系数和该第二系数大于一。其中，该第一系数和第二系数均可以由相关技术人员预先设置， 本申请实施例对其取值不作限定。
例如，以第一系数为10％，第二系数为50％为例对上述两种情况进行说明，对于情况一，情况一为像素点比值小于目标数值的情况，可以称情况一中的裁剪方式为全包式裁剪(crop)，在这种情况中，例如，目标数值为3，第一目标区域和第二目标区域的像素点比值为2，小于目标数值，这样该第一目标区域和第二目标区域的尺寸大小相差不大，也就是像素点数量相对平衡，则对样本图像进行裁剪使得第二目标区域的像素点消失的可能性很小，则在基于第一分割结果确定了第一目标区域后，将第一目标区域的连通域范围(统计值)与第一系数得到的范围作为裁剪范围。因而，可以将第三图像中的第一目标区域作为中心，外扩第一系数，得到输入第二图像分割模块的第四图像。具体地，可以将连通域范围外扩第一系数的范围作为裁剪范围，例如，可以将连通域范围的高度外扩5％，宽度外扩5％。当然，该第一系数也可以为110％，可以将连通域范围和第一系数的乘积作为裁剪范围。
下面举一个具体示例对情况一进行详细说明，参见图5，以脑部肿瘤分割示例，三种组织从外层到内层分别为：水肿(表示为edema)，非增强型肿瘤(表示为active)，增强型肿瘤(表示为necrotic)，CC表示外接标签方框，用于表示裁剪范围。根据统计结果，可以得知三种标签样本数目相对平衡，也就是脑部图像中三个区域之间的像素点比值均小于目标数值，像素点数目相对平衡。以上一模块输出对应类型的目标区域为中心，以该目标区域的连通域范围裁剪图像，也就是，以连通域范围外扩10％，作为下一层模型的输入。如第一个模块中输入的为全图，该模块主要分割edema，以预测图中edema CC1为中心，edema连通域范围(第一目标区域的连通域范围)外扩10％为区域裁剪图像，得到active模型的输入，后面的模块以此类推。上述裁剪过程中，如果裁剪范围超过当前模块输入的图像的范围则以完整全图区域补全，超过完整图区域的则丢弃。
对于情况二，情况二为像素点比值大于目标数值的情况，例如，目标数值为3，第一目标区域和第二目标区域的像素点比值为4，大于目标数值，这样该第一目标区域和第二目标区域的尺寸大小相差较大，也就是像素点数量失衡，第二目标区域相较于第一目标区域很小，属于小样本区域。因此，裁剪范围不能以第一目标区域的连通域范围为准，否则第二目标区域所占比例太小，易丢失，裁剪范围也不能完全以第二目标区域的连通域范围为准，可以补充上下文 信息进行扩充，因此，可以外扩50％。
下面举一个具体示例对情况二进行详细说明，参见图6，以肝癌为例。肝部图像的组织从外层到内层分别为肝(表示为liver)和肝癌(表示为tumor)。liver即为第一目标区域，tumor即为第二目标区域。根据统计结果，liver和tumor样本失衡，也就是第一目标区域和第二目标区域的像素点比值大于用户容忍度(目标数值)，则liver属于大样本标签，tumor属于小样本标签。因此，liver模块在裁剪图像时，裁剪范围为连通域范围外扩50％，可以称这种裁剪方式为扫描式crop。第一个模块中输入的为全图，第一个模块分割liver和tumor，重点为liver，预测结果有两个连通域，其中Liver CC1大于tumor的训练集连通域范围，Liver CC2则小于此范围。以Liver CC1为中心，按照liver训练集统计范围外扩10％crop图像得到Liver Crop，该Liver Crop是在第一目标区域的基础上补入上下文信息后待进行裁剪的图像，接着以tumor连通域范围(第二目标区域的连通域范围)外扩50％的范围从上到下，从左到右扫描输入的图像得到Tumor模型的输入1，2，3。以Liver CC2为中心，tumor连通域范围外扩50％crop则可以得到tumor模型的第4个输入图。
305、计算机设备中的初始模型基于对多个分割结果和该样本图像的标签，获取每个分割结果的分割误差。
初始模型可以采用损失函数获取每个分割结果的分割误差，在一些实施例中，在该初始模型的训练过程中获取每个图像分割模块的分割结果的分割误差时，该图像分割模块对应的类型的权重大于其他类型的权重。这样通过损失函数的权重设置，可以使得每个图像分割模块更着重分割对应类型。例如，对于第一图像分割模块，该第一图像分割模块的分割结果在获取分割误差时，损失函数中，第一类型的权重大于其他类型的权重，例如，以四分类为例，第一类型的权重为0.7，其他三类的权重均为0.1。
306、计算机设备基于多个分割误差，对初始模型的模型参数进行调整，直至达到预设条件时停止，得到图像分割模型，该初始模型的模型参数至少包括每个图像分割模块的模块参数。
计算机设备在获取到多个分割误差后，则可以基于该多个分割误差对初始模型的模型参数进行调整，通过多次调整后的模型参数可以使得该初始模型再对样本图像进行分割时，分割误差减小，也就是准确性更高，这样达到预设条件时，模型就训练完成。其中，每次对样本图像进行分割，获取分割误差和模 型参数调整过程均为一次迭代过程，上述模型训练过程即为多次迭代过程。
在一个具体的可能实施例中，上述初始模型的模型参数还可以包括每个图像分割模块的权重，也就是该初始模型的输出为综合该多个图像分割模块的分割结果的输出，该输出可以为对多个图像分割模块的加权求和结果，也就是在多个图像分割模块得到多个分割结果后，可以基于对多个分割结果进行加权求和，输出分割后的图像。需要说明的是，上述各个图像分割模块可以分别训练后，再训练该每个图像分割模块的权重，也可以在对每个图像分割模块的模块参数进行调整时，也调整每个图像分割模块的权重，本申请实施例对此不作限定。
例如，相关技术中，对于多模型融合，通常通过对概率取平均或采用投票的方式实现，而本申请实施例中，该多个图像分割模块的权重可以采用Dice值，Dice值为一种针对分割算法的评价指标。其中，该Dice值的取值范围可以为[0，1]，该Dice值越大，则表示分割性能越好，该Dice值可以基于交叉验证的方式确定。
该预设条件可以基于梯度下降法确定，例如，可以为分割误差收敛，也可以为迭代次数达到目标次数。在一些实施例中，上述图像分割模块的权重和预设条件可以基于交叉验证的方式确定。具体地，该第一迭代停止次数可以基于k-折交叉验证的方式确定，例如，可以基于五折交叉验证的方式确定。以五折交叉验证为例，可以将样本图像分为五部分，将其中四部分作为训练集，将另外一部分作为验证集，再以另外的组合方式进行多次训练和验证，当然，也可以确定不同组合方式后，同时以不同的组合方式对初始模型进行训练和验证，这样通过对样本数据的多种组合进行训练和验证，使得该初始模型遍历了所有的样本数据，训练后的模型的通用性更好，分割结果更准确。其中，该交叉验证过程主要为每进行一定次数的迭代过程时，通过验证数据对训练的模型进行验证，如果分割误差符合预设条件，则可以停止，如果不符合，则可以继续进行上述迭代过程，本申请实施例在此不作过多赘述。
上述步骤303至步骤306为基于该多个样本图像，对该初始模型进行训练，得到图像分割模型的过程，在上述模型训练过程中，初始模型可以自行根据样本图像进行分析，确定图像分割模块的数量，从而可以适用于多种场景，更具通用性、实用性和适用性。
上述训练得到图像分割模型后，当获取到待分割的第一图像时，计算机设 备调用该图像分割模型，由该图像分割模型基于多个图像分割模块，对该第一图像进行分割，输出第二图像，该第一图像也是目标人体组织图像。
需要说明的是，如果图像分割模块采用2D网络作为基础网络，输入的样本图像还可能是3D图像，则该初始模型还可以将3D图像转化为2D图像序列，也即是获取3D图像的切片，将切片输入图像分割模块中进行分割。
在一些实施例中，该初始模型和该图像分割模型均包括三个视角子模型，该三个视角子模型分别用于按照不同视角获取图像的切片和对图像进行分割。例如，如图7所示，该三个视角子模型可以分别按照X轴、Y轴、Z轴获取图像的切片，并分别进行分割，最终融合三个视角子模型的分割结果，输出分割后的图像。
这样通过不同视角获取图像的切片，从而综合不同视角的分割结果，可以提高图像分割模型对图像分割的准确性。相应地，初始模型在确定图像分割模块的数量时，可以将该多个样本图像的前景像素点的类型数量作为每个视角子模型中的图像分割模块的数量。也就是，每个视角子模型中的图像分割模块的数量均为前景像素点的类型数量。上述模型训练过程中，对于每个视角子模型，均可以基于多个图像分割模块依次对样本图像进行分割，从而综合该三个视角子模型的多个分割结果，得到最终的图像。其中，该三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果。
在一些实施例中，每个视角子模型也可以分别对应有权重，也就是上述Dice值，该每个视角子模型的权重也可以基于交叉验证的方式确定。例如，如图8所示，以四分类为例，每个视角子模型中包括三个图像分割模块，分别为：Model A、Model B和Model C。该三个模块依次对图像进行分割，得到三个阶段的分割结果，则在融合该三个分割结果时，可以基于三个模块的权重(Dice值)，对分割结果进行加权求和，得到某个视角子模型的分割结果。当然，在此仅以一个视角子模型为例进行说明，在综合多个视角子模型的分割结果时，可以也考虑到每个视角子模型的权重，得到最终输出的图像。
相应地，基于训练得到的图像分割模型对第一图像进行分割的过程中，计算机设备可以由该三个视角子模型分别按照对应的视角，获取第一图像的至少一个切片，由每个视角子模型中的多个图像分割模块对每个切片进行分割，基于该三个视角子模型的分割结果，输出第二图像。其中，该三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果。基于该三 个视角子模型的分割结果，输出第二图像的过程与训练时同理，可以为：计算机设备基于该三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重，对该三个视角子模型的多个图像分割模块的图像分割结果进行加权求和，输出第二图像，本申请实施例在此不多做赘述。
本申请实施例通过基于样本图像对初始模型进行训练得到图像分割模型，从而在获取到第一图像时，可以基于训练得到的图像分割模型对第一图像进行分割，其中，初始模型可以自行基于样本图像的像素点的类型数量确定图像分割模块的数量，因而对于不同人体组织图像均可以直接在该初始模型的基础上进行训练，而无需人工参与，重新设计模型，因此，上述图像分割模型的通用性、适用性和实用性好。进一步地，上述方法对于一切嵌套式临床组织结构分割具有通用性和针对性，分割性能和时效得到有效提升。且初始模型的结构的可变性使得该方法的可扩展性十分强。
图9是本申请实施例提供的一种图像分割方法的流程图，该图像分割方法应用于计算机设备中，该计算机设备可以为上述实施环境中的计算机设备201。在本申请实施例中，主要对当获取到待分割的第一图像时，调用该图像分割模型，由该图像分割模型基于多个图像分割模块，对该第一图像进行分割，输出第二图像的过程进行了详细说明。参见图9，该图像分割方法可以包括以下步骤：
901、计算机设备获取待分割的第一图像。
计算机设备在检测到图像分割操作时执行该步骤901，也可以接收用户导入的待分割的第一图像，还可以接收其他计算机设备发送的图像分割请求，该图像分割请求中携带有待分割的第一图像，从该图像分割请求中提取待分割的第一图像，或该图像分割请求中可以携带有该第一图像的相关信息，计算机设备可以基于该相关信息，执行该步骤901，当然，该计算机设备也可以通过成像原理获取得到待分割的第一图像。本申请实施例对该待分割的第一图像的具体获取方式和获取时机不作限定。
例如，其他计算机设备可以通过成像原理获取得到待分割的第一图像，并向该计算机设备发送该待分割的第一图像，该计算机设备获取到该待分割的第一图像，该第一图像可以为上述目标人体组织图像，这样可以执行下述步骤，利用通过该目标人体组织的样本图像进行训练得到的图像分割模型，对该第一图像进行分割。
902、计算机设备调用图像分割模型。
其中，该图像分割模型包括多个图像分割模块。该多个图像分割模块的数量在上述图3所示实施例中，对初始模型进行训练时，由初始模型确定。不同的图像分割模块用于对图像的不同区域进行分割，该多个图像分割模块可以依次对第一图像进行分割，实现流式分割方案。
该计算机设备中可以预先存储有图像分割模型，在一些实施例中，该计算机设备即为图3所示的计算机设备，也就是该计算机设备上存储的图像分割模型即为在该计算机设备上训练得到的。在一些实施例中，该计算机设备不是图3所示的计算机设备，可以是在其他计算机设备上训练得到图像分割模型，该计算机设备可以从其他计算机设备上获取该训练好的图像分割模型。当然，该计算机设备上也可以没有存储有图像分割模型，在该计算机设备获取到待分割的第一图像，需要对第一图像进行分割时，可以实时从其他计算机设备处调用图像分割模型，本申请实施例对此不作限定。
与上述步骤306中的内容同理，该图像分割模型还可以包括三个视角子模型，该三个视角子模型分别用于按照不同视角获取图像的切片和对图像进行分割。相应地，每个视角子模型中包括多个图像分割模块。该多个图像分割模块分别对应像素点的一种类型，也就是多个图像分割模块分别用于着重分割像素点的一种类型。
903、计算机设备将该第一图像输入该图像分割模型中，由该图像分割模型基于多个图像分割模块，对该第一图像进行分割，得到多个分割结果。
与上述步骤306中的内容同理，如果该图像分割模型包括三个视角子模型，则该步骤903可以为：计算机设备将该第一图像输入图像分割模型中，由该图像分割模型中的三个视角子模型分别按照对应的视角，获取第一图像的至少一个切片，由每个视角子模型中的多个图像分割模块对每个切片进行分割，基于该三个视角子模型的分割结果，输出第二图像。
与步骤304中的内容同理，每个图像分割模块可以对图像进行分割、裁剪，得到分割结果，具体地，对于该多个图像分割模块中相邻的两个图像分割模块，计算机设备可以基于第一图像分割模块对该第三图像进行分割，得到第一分割结果，该第一图像分割模块为该相邻的两个图像分割模块中顺序在前的图像分割模块。计算机设备基于第二图像分割模块对基于该第一分割结果裁剪得到的第四图像进行分割，得到第二分割结果，该第二图像分割模块为该相邻的两个 图像分割模块中顺序在后的图像分割模块，该第四图像为该第三图像的部分区域。其中，该第四图像包括该第一分割结果所指示的第一类型的像素点，该第一类型为该第一图像分割模块对应的类型。
904、计算机设备中的图像分割模型基于多个分割结果，输出第二图像。
在包括三个视角子模型的情况中，该三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果。相应地，图像分割模型基于该三个视角子模型的分割结果，输出第二图像的过程中，也可以基于该三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重，对该三个视角子模型的多个图像分割模块的图像分割结果进行加权求和，输出第二图像。其中，该三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重基于交叉验证的方式确定。上述内容均与图3所示实施例中的相关内容同理，本申请实施例在此不多做赘述。
在该步骤904后，在得到第二图像之后，计算机设备可以存储该第二图像，当然，也可以将第一图像和第二图像对应存储，如果该计算机设备为基于其他计算机设备的图像分割请求进行的上述图像分割过程，也可以将该第二图像发送至该其他计算机设备，本申请实施例对此不作限定。
本申请实施例通过基于样本图像对初始模型进行训练得到图像分割模型，从而在获取到第一图像时，可以基于训练得到的图像分割模型对第一图像进行分割，其中，初始模型可以自行基于样本图像的像素点的类型数量确定图像分割模块的数量，因而对于不同人体组织图像均可以直接在该初始模型的基础上进行训练，而无需人工参与，重新设计模型，因此，上述图像分割模型的通用性、适用性和实用性好。
上述所有可选技术方案，可以采用任意结合形成本申请的可选实施例，在此不再一一赘述。
应该理解的是，本申请各实施例中的各个步骤并不是必然按照步骤标号指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，各实施例中至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的 至少一部分轮流或者交替地执行。
在一个实施例中，还提供了一种计算机设备，该计算机设备包括图像分割装置，图像分割装置中包括各个模块，每个模块可全部或部分通过软件、硬件或其组合来实现。
图10是本申请实施例提供的一种图像分割装置的结构示意图，参见图10，该装置包括：
获取模块1001，用于获取多个样本图像；
训练模块1002，用于调用初始模型，将该多个样本图像输入初始模型中，基于该多个样本图像，对该初始模型进行训练，得到图像分割模型，该初始模型用于根据该多个样本图像的像素点的类型数量确定图像分割模块的数量，不同的图像分割模块用于对图像的不同区域进行分割；
分割模块1003，用于当获取到待分割的第一图像时，调用该图像分割模型，由该图像分割模型基于多个图像分割模块，对该第一图像进行分割，输出第二图像，该多个样本图像和该第一图像均为目标人体组织图像。
在一些实施例中，该训练模块1002用于将该多个样本图像的前景像素点的类型数量作为该初始模型中的图像分割模块的数量。
在一些实施例中，每个样本图像携带有标签，该标签用于指示该样本图像的目标分割结果；
相应地，该训练模块1002还用于对该多个样本图像的标签进行分析，得到该多个样本图像的像素点的类型数量。
在一些实施例中，该分割模块1003用于：
对于该多个图像分割模块中相邻的两个图像分割模块，基于第一图像分割模块对第三图像进行分割，得到第一分割结果，该第一图像分割模块为该相邻的两个图像分割模块中顺序在前的图像分割模块；
基于第二图像分割模块对基于该第一分割结果裁剪得到的第四图像进行分割，得到第二分割结果，该第二图像分割模块为该相邻的两个图像分割模块中顺序在后的图像分割模块，该第四图像为该第三图像的部分区域。
在一些实施例中，该多个图像分割模块分别对应像素点的一种类型；该第四图像包括该第一分割结果所指示的第一类型的像素点，该第一类型为该第一图像分割模块对应的类型；在该初始模型的训练过程中获取每个图像分割模块 的分割结果的分割误差时，该图像分割模块对应的类型的权重大于其他类型的权重。
在一些实施例中，该获取模块1001还用于在该初始模型的训练过程中，获取该多个样本图像对应的多个目标区域中相邻目标区域之间的像素点比值，该目标区域为该多个样本图像中目标类型的像素点所在区域；
该装置还包括：
裁剪模块，用于在该多个图像分割模块中每个图像分割模块对图像进行裁剪时，根据该多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，对图像进行裁剪。
在一些实施例中，该获取模块1001还用于在该初始模型的训练过程中，获取该多个样本图像对应的多个目标区域的连通域范围；
该裁剪模块用于在该多个图像分割模块中每个图像分割模块对图像进行裁剪时，根据该多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，以及该多个目标区域的连通域范围，对图像进行裁剪。
在一些实施例中，该裁剪模块用于：
对于该第一图像分割模块，根据第一目标区域与第二目标区域的像素点比值与目标数值的大小关系，以及该第一目标区域或第二目标区域的连通域范围，对该第三图像进行裁剪，该第一目标区域为该第一图像分割模块对应的第一类型的像素点所在区域，该第二目标区域为该第二图像分割模块对应的第二类型的像素点所在区域。
在一些实施例中，该裁剪模块还用于：
当该第一目标区域与第一目标区域的像素点比值小于目标数值时，基于该第一目标区域的连通域范围，对该第三图像进行裁剪；
当该第一目标区域与第二目标区域的像素点比值大于或等于目标数值时，基于该第二目标区域的连通域范围，对该第三图像进行裁剪。
在一些实施例中，当基于该第二目标区域的连通域范围，对第三图像进行裁剪时，裁剪后的第四图像的尺寸基于该连通域范围和第一系数得到；
当基于该第二目标区域的连通域范围，对该第三图像进行裁剪时，裁剪后的第四图像的尺寸基于该第二目标区域的连通域范围和第二系数得到，该第一系数小于该第二系数。
在一些实施例中，该初始模型和该图像分割模型均包括三个视角子模型， 该三个视角子模型分别用于按照不同视角获取图像的切片和对图像进行分割；
相应地，该训练模块1002用于将该多个样本图像的前景像素点的类型数量作为每个视角子模型中的图像分割模块的数量；
相应地，该分割模块1003用于由该三个视角子模型分别按照对应的视角，获取第一图像的至少一个切片，由每个视角子模型中的多个图像分割模块对每个切片进行分割，基于该三个视角子模型的分割结果，输出第二图像。
在一些实施例中，该三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果；
相应地，该分割模块1003还用于基于该三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重，对该三个视角子模型的多个图像分割模块的图像分割结果进行加权求和，输出第二图像，该三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重基于交叉验证的方式确定。
在一些实施例中，该训练模块1002还用于在该初始模型的训练过程中，对于每个图像分割模块，根据该多个样本图像的图像数量，获取该图像数量对应的图像分割子模块作为该图像分割模块，该图像分割模块中包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
本申请实施例提供的装置，通过基于样本图像对初始模型进行训练得到图像分割模型，从而在获取到第一图像时，可以基于训练得到的图像分割模型对第一图像进行分割，其中，初始模型可以自行基于样本图像的像素点的类型数量确定图像分割模块的数量，因而对于不同人体组织图像均可以直接在该初始模型的基础上进行训练，而无需人工参与，重新设计模型，因此，上述图像分割模型的通用性、适用性和实用性好。
需要说明的是：上述实施例提供的图像分割装置在分割图像时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将计算机设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的图像分割装置与图像分割方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
上述计算机设备可以被提供为下述图11所示的终端，也可以被提供为下述 图12所示的服务器，本申请实施例对此不作限定。
图11是本申请实施例提供的一种终端的结构示意图。该终端1100可以是：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑或台式电脑。终端1100还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1100包括有：处理器1101和存储器1102。
处理器1101可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器1101可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器1101也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器1101可以集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器1101还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1102可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1102还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1102中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1101所执行以实现本申请中方法实施例提供的图像分割模型训练方法或图像分割方法。
在一些实施例中，终端1100还可选包括有：外围设备接口1103和至少一个外围设备。处理器1101、存储器1102和外围设备接口1103之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口1103相连。具体地，外围设备包括：射频电路1104、触摸显示屏1105、摄像头1106、音频电路1107、定位组件1108和电源1109中的至少一种。
外围设备接口1103可被用于将I/O(Input/Output，输入/输出)相关的至少 一个外围设备连接到处理器1101和存储器1102。在一些实施例中，处理器1101、存储器1102和外围设备接口1103被集成在同一芯片或电路板上；在一些其他实施例中，处理器1101、存储器1102和外围设备接口1103中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路1104用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路1104通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1104将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路1104包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路1104可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1104还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1105用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏1105是触摸显示屏时，显示屏1105还具有采集在显示屏1105的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1101进行处理。此时，显示屏1105还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏1105可以为一个，设置终端1100的前面板；在另一些实施例中，显示屏1105可以为至少两个，分别设置在终端1100的不同表面或呈折叠设计；在一些实施例中，显示屏1105可以是柔性显示屏，设置在终端1100的弯曲表面上或折叠面上。甚至，显示屏1105还可以设置成非矩形的不规则图形，也即异形屏。显示屏1105可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode，有机发光二极管)等材质制备。
摄像头组件1106用于采集图像或视频。可选地，摄像头组件1106包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端的前面板，后置摄像头设置在终端的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些 实施例中，摄像头组件1106还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路1107可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器1101进行处理，或者输入至射频电路1104以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端1100的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器1101或射频电路1104的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路1107还可以包括耳机插孔。
定位组件1108用于定位终端1100的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件1108可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源1109用于为终端1100中的各个组件进行供电。电源1109可以是交流电、直流电、一次性电池或可充电电池。当电源1109包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1100还包括有一个或多个传感器1110。该一个或多个传感器1110包括但不限于：加速度传感器1111、陀螺仪传感器1112、压力传感器1113、指纹传感器1114、光学传感器1115以及接近传感器1116。
加速度传感器1111可以检测以终端1100建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器1111可以用于检测重力加速度在三个坐标轴上的分量。处理器1101可以根据加速度传感器1111采集的重力加速度信号，控制触摸显示屏1105以横向视图或纵向视图进行用户界面的显示。加速度传感器1111还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器1112可以检测终端1100的机体方向及转动角度，陀螺仪传感器1112可以与加速度传感器1111协同采集用户对终端1100的3D动作。处理器1101根据陀螺仪传感器1112采集的数据，可以实现如下功能：动作感应 (比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器1113可以设置在终端1100的侧边框和/或触摸显示屏1105的下层。当压力传感器1113设置在终端1100的侧边框时，可以检测用户对终端1100的握持信号，由处理器1101根据压力传感器1113采集的握持信号进行左右手识别或快捷操作。当压力传感器1113设置在触摸显示屏1105的下层时，由处理器1101根据用户对触摸显示屏1105的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器1114用于采集用户的指纹，由处理器1101根据指纹传感器1114采集到的指纹识别用户的身份，或者，由指纹传感器1114根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器1101授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器1114可以被设置终端1100的正面、背面或侧面。当终端1100上设置有物理按键或厂商Logo时，指纹传感器1114可以与物理按键或厂商Logo集成在一起。
光学传感器1115用于采集环境光强度。在一个实施例中，处理器1101可以根据光学传感器1115采集的环境光强度，控制触摸显示屏1105的显示亮度。具体地，当环境光强度较高时，调高触摸显示屏1105的显示亮度；当环境光强度较低时，调低触摸显示屏1105的显示亮度。在另一个实施例中，处理器1101还可以根据光学传感器1115采集的环境光强度，动态调整摄像头组件1106的拍摄参数。
接近传感器1116，也称距离传感器，通常设置在终端1100的前面板。接近传感器1116用于采集用户与终端1100的正面之间的距离。在一个实施例中，当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变小时，由处理器1101控制触摸显示屏1105从亮屏状态切换为息屏状态；当接近传感器1116检测到用户与终端1100的正面之间的距离逐渐变大时，由处理器1101控制触摸显示屏1105从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图11中示出的结构并不构成对终端1100的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图12是本申请实施例提供的一种服务器的结构示意图，该服务器1200可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器(central processing units，CPU)1201和一个或一个以上的存储器1202，其中，所述存储器1202中存储有至少一条指令，所述至少一条指令由所述处理器1201加载并执行以实现上述各个方法实施例提供的图像分割模型训练方法或图像分割方法。当然，该服务器还可以具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该服务器还可以包括其他用于实现设备功能的部件，在此不做赘述。
上述计算机设备可以被提供为下述图13所示的服务器。如图13所示，该服务器包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储图像数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像分割方法或图像分割模型训练方法。
上述计算机设备可以被提供为下述图14所示的终端。如图14所示，该终端包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像分割方法或图像分割模型训练方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图13、14示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的服务器、终端的限定，具体的服务器、终端可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，本申请提供的图像分割装置可以实现为一种计算机可读指令的形式，计算机可读指令可在如图13所示的服务器上运行，也可以在如图14所示的终端上运行。服务器或终端的存储器中可存储组成该图像分割装置的各个程序模块，比如获取模块1001、训练模块1002和分割模块1003。各个程序模块构成的计算机可读指令使得处理器执行本说明书中描述的本申请各个实施例的图像分割方法或图像分割模型训练方法中的步骤。
本申请实施例提供了一种计算机可读存储介质，所述存储介质中存储有计算机可读指令，该计算机可读指令由处理器加载并具有以实现上述实施例的图像分割方法或图像分割模型训练方法中所具有的操作。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
本领域技术人员在考虑说明书及实践这里公开的申请后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (26)

  1. 一种图像分割方法，其特征在于，所述方法包括：
    计算机设备获取多个样本图像；
    所述计算机设备调用初始模型，将所述多个样本图像输入初始模型中，基于所述多个样本图像，对所述初始模型进行训练，得到图像分割模型，所述初始模型用于根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，不同的图像分割模块用于对图像的不同区域进行分割；
    当获取到待分割的第一图像时，所述计算机设备调用所述图像分割模型，由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，输出第二图像，所述多个样本图像和所述第一图像均为目标人体组织图像。
  2. 根据权利要求1所述的方法，其特征在于，所述计算机设备根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，包括：
    所述计算机设备将所述多个样本图像的前景像素点的类型数量作为所述初始模型中的图像分割模块的数量。
  3. 根据权利要求1所述的方法，其特征在于，每个样本图像携带有标签，所述标签用于指示所述样本图像的目标分割结果；
    相应地，所述计算机设备根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量之前，所述方法还包括：
    所述计算机设备对所述多个样本图像的标签进行分析，得到所述多个样本图像的像素点的类型数量。
  4. 根据权利要求1所述的方法，其特征在于，所述由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，包括：
    对于所述多个图像分割模块中相邻的两个图像分割模块，所述计算机设备基于第一图像分割模块对第三图像进行分割，得到第一分割结果，所述第一图像分割模块为所述相邻的两个图像分割模块中顺序在前的图像分割模块；
    所述计算机设备基于第二图像分割模块对基于所述第一分割结果裁剪得到的第四图像进行分割，得到第二分割结果，所述第二图像分割模块为所述相邻的两个图像分割模块中顺序在后的图像分割模块，所述第四图像为所述第三图 像的部分区域。
  5. 根据权利要求4所述的方法，其特征在于，所述多个图像分割模块分别对应像素点的一种类型；所述第四图像包括所述第一分割结果所指示的第一类型的像素点，所述第一类型为所述第一图像分割模块对应的类型；在所述初始模型的训练过程中获取每个图像分割模块的分割结果的分割误差时，所述图像分割模块对应的类型的权重大于其他类型的权重。
  6. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    在所述初始模型的训练过程中，所述计算机设备获取所述多个样本图像对应的多个目标区域中相邻目标区域之间的像素点比值，所述目标区域为所述多个样本图像中目标类型的像素点所在区域；
    在所述多个图像分割模块中每个图像分割模块对图像进行裁剪时，所述计算机设备根据所述多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，对图像进行裁剪。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    在所述初始模型的训练过程中，所述计算机设备获取所述多个样本图像对应的多个目标区域的连通域范围；
    所述在所述多个图像分割模块中每个图像分割模块对图像进行裁剪时，所述计算机设备根据所述多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，以及所述多个目标区域的连通域范围，对图像进行裁剪。
  8. 根据权利要求7所述的方法，其特征在于，所述在所述多个图像分割模块中每个图像分割模块对图像进行裁剪时，所述计算机设备根据所述多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，以及所述多个目标区域的连通域范围，对图像进行裁剪，包括：
    对于所述第一图像分割模块，所述计算机设备根据第一目标区域与第二目标区域的像素点比值与目标数值的大小关系，以及所述第一目标区域或所述第二目标区域的连通域范围，对所述第三图像进行裁剪，所述第一目标区域为所述第一图像分割模块对应的第一类型的像素点所在区域，所述第二目标区域为 所述第二图像分割模块对应的第二类型的像素点所在区域。
  9. 根据权利要求8所述的方法，其特征在于，所述根据第一目标区域与第二目标区域的像素点比值与目标数值的大小关系，以及所述第一目标区域或所述第二目标区域的连通域范围，对所述第三图像进行裁剪，包括：
    当所述第一目标区域与第二目标区域的像素点比值小于目标数值时，所述计算机设备基于所述第一目标区域的连通域范围，对所述第三图像进行裁剪，裁剪后的第四图像的尺寸基于所述第一目标区域的连通域范围和第一系数得到；
    当所述第一目标区域与第二目标区域的像素点比值大于或等于目标数值时，所述计算机设备基于所述第二目标区域的连通域范围，对所述第三图像进行裁剪，裁剪后的第四图像的尺寸基于所述第二目标区域的连通域范围和第二系数得到，所述第一系数小于所述第二系数。
  10. 根据权利要求1-9任一项所述的方法，其特征在于，所述初始模型和所述图像分割模型均包括三个视角子模型，所述三个视角子模型分别用于按照不同视角获取图像的切片和对图像进行分割；
    相应地，所述计算机设备根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，包括：
    所述计算机设备将所述多个样本图像的前景像素点的类型数量作为每个视角子模型中的图像分割模块的数量；
    相应地，所述由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，输出第二图像，包括：
    所述计算机设备由所述三个视角子模型分别按照对应的视角，获取第一图像的至少一个切片，由每个视角子模型中的多个图像分割模块对每个切片进行分割，基于所述三个视角子模型的分割结果，输出第二图像。
  11. 根据权利要求10所述的方法，其特征在于，所述三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果；
    相应地，所述基于所述三个视角子模型的分割结果，输出第二图像，包括：
    所述计算机设备基于所述三个视角子模型对应的权重和每个视角子模型中 每个图像分割模块对应的权重，对所述三个视角子模型的多个图像分割模块的图像分割结果进行加权求和，输出第二图像，所述三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重基于交叉验证的方式确定。
  12. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    在所述初始模型的训练过程中，对于每个图像分割模块，所述计算机设备根据所述多个样本图像的图像数量，获取所述图像数量对应的图像分割子模块作为所述图像分割模块，所述图像分割模块中包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
  13. 一种图像分割装置，其特征在于，所述装置包括：
    获取模块，用于获取多个样本图像；
    训练模块，用于调用初始模型，将所述多个样本图像输入初始模型中，基于所述多个样本图像，对所述初始模型进行训练，得到图像分割模型，所述初始模型用于根据所述多个样本图像的像素点的类型数量确定图像分割模块的数量，不同的图像分割模块用于对图像的不同区域进行分割；
    分割模块，用于当获取到待分割的第一图像时，调用所述图像分割模型，由所述图像分割模型基于多个图像分割模块，对所述第一图像进行分割，输出第二图像，所述多个样本图像和所述第一图像均为目标人体组织图像。
  14. 根据权利要求13所述的装置，其特征在于，所述训练模块还用于将所述多个样本图像的前景像素点的类型数量作为所述初始模型中的图像分割模块的数量。
  15. 根据权利要求13所述的装置，其特征在于，每个样本图像携带有标签，所述标签用于指示所述样本图像的目标分割结果，所述训练模块还用于对所述多个样本图像的标签进行分析，得到所述多个样本图像的像素点的类型数量。
  16. 根据权利要求13所述的装置，其特征在于，所述分割模块还用于对于所述多个图像分割模块中相邻的两个图像分割模块，基于第一图像分割模块对第三图像进行分割，得到第一分割结果，所述第一图像分割模块为所述相邻的 两个图像分割模块中顺序在前的图像分割模块；基于第二图像分割模块对基于所述第一分割结果裁剪得到的第四图像进行分割，得到第二分割结果，所述第二图像分割模块为所述相邻的两个图像分割模块中顺序在后的图像分割模块，所述第四图像为所述第三图像的部分区域。
  17. 根据权利要求16所述的装置，其特征在于，所述多个图像分割模块分别对应像素点的一种类型；所述第四图像包括所述第一分割结果所指示的第一类型的像素点，所述第一类型为所述第一图像分割模块对应的类型；在所述初始模型的训练过程中获取每个图像分割模块的分割结果的分割误差时，所述图像分割模块对应的类型的权重大于其他类型的权重。
  18. 根据权利要求16所述的装置，其特征在于，所述获取模块还用于在所述初始模型的训练过程中，获取所述多个样本图像对应的多个目标区域中相邻目标区域之间的像素点比值，所述目标区域为所述多个样本图像中目标类型的像素点所在区域；所述装置还包括：
    裁剪模块，用于在所述多个图像分割模块中每个图像分割模块对图像进行裁剪时，根据所述多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，对图像进行裁剪。
  19. 根据权利要求18所述的装置，其特征在于，所述获取模块还用于在所述初始模型的训练过程中，获取所述多个样本图像对应的多个目标区域的连通域范围；所述装置还包括：
    裁剪模块，用于在所述多个图像分割模块中每个图像分割模块对图像进行裁剪时，根据所述多个目标区域中相邻目标区域之间的像素点比值与目标数值的大小关系，以及所述多个目标区域的连通域范围，对图像进行裁剪。
  20. 根据权利要求19所述的装置，其特征在于，所述裁剪模块还用于对于所述第一图像分割模块，根据第一目标区域与第二目标区域的像素点比值与目标数值的大小关系，以及所述第一目标区域或所述第二目标区域的连通域范围，对所述第三图像进行裁剪，所述第一目标区域为所述第一图像分割模块对应的第一类型的像素点所在区域，所述第二目标区域为所述第二图像分割模块对应 的第二类型的像素点所在区域。
  21. 根据权利要求20所述的装置，其特征在于，所述裁剪模块还用于当所述第一目标区域与第二目标区域的像素点比值小于目标数值时，基于所述第一目标区域的连通域范围，对所述第三图像进行裁剪，裁剪后的第四图像的尺寸基于所述第一目标区域的连通域范围和第一系数得到；当所述第一目标区域与第二目标区域的像素点比值大于或等于目标数值时，基于所述第二目标区域的连通域范围，对所述第三图像进行裁剪，裁剪后的第四图像的尺寸基于所述第二目标区域的连通域范围和第二系数得到，所述第一系数小于所述第二系数。
  22. 根据权利要求13至21任一项所述的装置，其特征在于，所述初始模型和所述图像分割模型均包括三个视角子模型，所述三个视角子模型分别用于按照不同视角获取图像的切片和对图像进行分割；所述训练模块还用于将所述多个样本图像的前景像素点的类型数量作为每个视角子模型中的图像分割模块的数量；所述分割模块还用于由所述三个视角子模型分别按照对应的视角，获取第一图像的至少一个切片，由每个视角子模型中的多个图像分割模块对每个切片进行分割，基于所述三个视角子模型的分割结果，输出第二图像。
  23. 根据权利要求22所述的装置，其特征在于，所述三个视角子模型中每个视角子模型的分割结果包括多个图像分割模块的图像分割结果；所述分割模块还用于基于所述三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重，对所述三个视角子模型的多个图像分割模块的图像分割结果进行加权求和，输出第二图像，所述三个视角子模型对应的权重和每个视角子模型中每个图像分割模块对应的权重基于交叉验证的方式确定。
  24. 根据权利要求13所述的装置，其特征在于，所述训练模块还用于在所述初始模型的训练过程中，对于每个图像分割模块，根据所述多个样本图像的图像数量，获取所述图像数量对应的图像分割子模块作为所述图像分割模块，所述图像分割模块中包括至少一个图像分割子模块，不同的图像分割子模块的深度不同。
  25. 一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如权利要求1至12任一项所述的方法。
  26. 一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如权利要求1至12任一项所述的方法。
PCT/CN2019/110557 2018-10-16 2019-10-11 图像分割方法、装置、计算机设备及存储介质 WO2020078268A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19873499.8A EP3869455A4 (en) 2018-10-16 2019-10-11 IMAGE SEGMENTATION PROCESS AND APPARATUS, COMPUTER DEVICE AND INFORMATION MEDIA
US17/039,925 US11403763B2 (en) 2018-10-16 2020-09-30 Image segmentation method and apparatus, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811204371.6A CN109360210B (zh) 2018-10-16 2018-10-16 图像分割方法、装置、计算机设备及存储介质
CN201811204371.6 2018-10-16

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/039,925 Continuation US11403763B2 (en) 2018-10-16 2020-09-30 Image segmentation method and apparatus, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2020078268A1 true WO2020078268A1 (zh) 2020-04-23

Family

ID=65349271

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110557 WO2020078268A1 (zh) 2018-10-16 2019-10-11 图像分割方法、装置、计算机设备及存储介质

Country Status (4)

Country Link
US (1) US11403763B2 (zh)
EP (1) EP3869455A4 (zh)
CN (1) CN109360210B (zh)
WO (1) WO2020078268A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102284A (zh) * 2020-09-14 2020-12-18 推想医疗科技股份有限公司 图像分割模型的训练样本的标记方法、训练方法及装置
CN113971427A (zh) * 2020-07-23 2022-01-25 四川大学 基于改进模型的岩屑识别方法

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109360210B (zh) * 2018-10-16 2019-10-25 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质
CN109934812B (zh) * 2019-03-08 2022-12-09 腾讯科技（深圳）有限公司 图像处理方法、装置、服务器及存储介质
CN112446893B (zh) * 2019-08-28 2023-06-02 北京精诊医疗科技有限公司 肝脏图像的轮廓分割方法和装置
CN110942447B (zh) * 2019-10-18 2023-07-21 平安科技（深圳）有限公司 Oct图像分割方法、装置、设备及存储介质
CN111447354B (zh) * 2019-10-23 2020-10-27 岳阳县辉通物联网科技有限公司 智能调整式摄像平台
CN110853128B (zh) * 2019-11-12 2023-03-21 腾讯科技（深圳）有限公司 虚拟物体显示方法、装置、计算机设备及存储介质
CN110910408A (zh) * 2019-11-28 2020-03-24 慧影医疗科技（北京）有限公司 图像分割方法、装置、电子设备及可读存储介质
CN111126494B (zh) * 2019-12-25 2023-09-26 中国科学院自动化研究所 基于各向异性卷积的图像分类方法及系统
CN111241204B (zh) * 2020-02-14 2023-01-10 苏州浪潮智能科技有限公司 一种梯度数据的同步方法、装置、设备及存储介质
CN111666949B (zh) * 2020-06-16 2023-06-20 南京大学 一种基于迭代分割的图像语义分割方法
CN111862096B (zh) * 2020-09-23 2021-06-18 平安科技（深圳）有限公司 图像分割方法、装置、电子设备及存储介质
CN112802032A (zh) * 2021-01-19 2021-05-14 上海商汤智能科技有限公司 图像分割网络的训练和图像处理方法、装置、设备及介质
CN113160232B (zh) * 2021-03-29 2022-01-28 吉林大学 基于MU-Net应用于CT影像的颅内出血病灶分割算法
CN113628325B (zh) * 2021-08-10 2024-03-26 海盐县南北湖医学人工智能研究院 一种小器官肿瘤演化的模型建立方法及计算机可读存储介质
CN114782911B (zh) * 2022-06-20 2022-09-16 小米汽车科技有限公司 图像处理的方法、装置、设备、介质、芯片及车辆
CN115937114B (zh) * 2022-11-25 2024-06-04 无锡海纳智能科技有限公司 一种风机图片的预处理方法及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105574859A (zh) * 2015-12-14 2016-05-11 中国科学院深圳先进技术研究院 一种基于ct图像的肝脏肿瘤分割方法及装置
CN107742122A (zh) * 2017-10-27 2018-02-27 浙江大华技术股份有限公司 一种x光图像的分割方法及装置
EP3373246A1 (en) * 2017-03-08 2018-09-12 Tsinghua University Inspection apparatuses and methods for segmenting an image of a vehicle
CN109360210A (zh) * 2018-10-16 2019-02-19 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100501728C (zh) * 2003-01-31 2009-06-17 佳能株式会社 图像处理方法、系统、程序、程序存储介质以及信息处理设备
US8369590B2 (en) * 2007-05-21 2013-02-05 Cornell University Method for segmenting objects in images
US8577115B2 (en) * 2008-03-04 2013-11-05 Tomotherapy Incorporated Method and system for improved image segmentation
US8885926B2 (en) * 2009-04-15 2014-11-11 Massachusetts Institute Of Technology Image and data segmentation
US9159129B2 (en) * 2011-07-12 2015-10-13 Definiens Ag Generating image-based diagnostic tests by optimizing image analysis and data mining of co-registered images
CN102360494B (zh) * 2011-10-18 2013-09-04 中国科学院自动化研究所 一种多前景目标交互式图像分割方法
WO2014152919A1 (en) * 2013-03-14 2014-09-25 Arizona Board Of Regents, A Body Corporate Of The State Of Arizona For And On Behalf Of Arizona State University Kernel sparse models for automated tumor segmentation
JP6629762B2 (ja) * 2014-05-23 2020-01-15 ベンタナ メディカル システムズ， インコーポレイテッド 画像内の生物学的構造及び／又はパターンの検出のためのシステム及び方法
CN104050666B (zh) * 2014-06-10 2017-07-11 电子科技大学 基于分割的脑部mr图像配准方法
WO2017091833A1 (en) 2015-11-29 2017-06-01 Arterys Inc. Automated cardiac volume segmentation
US10181188B2 (en) * 2016-02-19 2019-01-15 International Business Machines Corporation Structure-preserving composite model for skin lesion segmentation
CN106226050B (zh) * 2016-07-15 2019-02-01 北京航空航天大学 一种基于卷积神经网络的tfds故障图像自动识别方法
CN109690554B (zh) 2016-07-21 2023-12-05 西门子保健有限责任公司 用于基于人工智能的医学图像分割的方法和系统
US9965863B2 (en) * 2016-08-26 2018-05-08 Elekta, Inc. System and methods for image segmentation using convolutional neural network
US10354383B2 (en) * 2016-12-30 2019-07-16 Skinio, Llc Skin abnormality monitoring systems and methods
US10580131B2 (en) * 2017-02-23 2020-03-03 Zebra Medical Vision Ltd. Convolutional neural network for segmentation of medical anatomical images
CN107240102A (zh) * 2017-04-20 2017-10-10 合肥工业大学 基于深度学习算法的恶性肿瘤计算机辅助早期诊断方法
CN107506761B (zh) * 2017-08-30 2020-01-17 山东大学 基于显著性学习卷积神经网络的脑部图像分割方法及系统
CN107507201A (zh) 2017-09-22 2017-12-22 深圳天琴医疗科技有限公司 一种医学图像分割方法及装置
CN107977969B (zh) * 2017-12-11 2020-07-21 北京数字精准医疗科技有限公司 一种内窥镜荧光图像的分割方法、装置及存储介质
CN108492297B (zh) * 2017-12-25 2021-11-19 重庆师范大学 基于深度级联卷积网络的mri脑肿瘤定位与瘤内分割方法
CN108492301A (zh) * 2018-03-21 2018-09-04 广东欧珀移动通信有限公司 一种场景分割方法、终端及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105574859A (zh) * 2015-12-14 2016-05-11 中国科学院深圳先进技术研究院 一种基于ct图像的肝脏肿瘤分割方法及装置
EP3373246A1 (en) * 2017-03-08 2018-09-12 Tsinghua University Inspection apparatuses and methods for segmenting an image of a vehicle
CN107742122A (zh) * 2017-10-27 2018-02-27 浙江大华技术股份有限公司 一种x光图像的分割方法及装置
CN109360210A (zh) * 2018-10-16 2019-02-19 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3869455A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113971427A (zh) * 2020-07-23 2022-01-25 四川大学 基于改进模型的岩屑识别方法
CN113971427B (zh) * 2020-07-23 2023-08-18 四川大学 基于改进模型的岩屑识别方法
CN112102284A (zh) * 2020-09-14 2020-12-18 推想医疗科技股份有限公司 图像分割模型的训练样本的标记方法、训练方法及装置
CN112102284B (zh) * 2020-09-14 2024-05-28 推想医疗科技股份有限公司 图像分割模型的训练样本的标记方法、训练方法及装置

Also Published As

Publication number Publication date
US20210019890A1 (en) 2021-01-21
EP3869455A1 (en) 2021-08-25
EP3869455A4 (en) 2021-12-08
CN109360210A (zh) 2019-02-19
CN109360210B (zh) 2019-10-25
US11403763B2 (en) 2022-08-02

Similar Documents

Publication Publication Date Title
WO2020078268A1 (zh) 图像分割方法、装置、计算机设备及存储介质
JP7085062B2 (ja) 画像セグメンテーション方法、装置、コンピュータ機器およびコンピュータプログラム
US11710351B2 (en) Action recognition method and apparatus, and human-machine interaction method and apparatus
CN108960209B (zh) 身份识别方法、装置及计算机可读存储介质
CN110210571B (zh) 图像识别方法、装置、计算机设备及计算机可读存储介质
WO2020253657A1 (zh) 视频片段定位方法、装置、计算机设备及存储介质
US20220076000A1 (en) Image Processing Method And Apparatus
JP7058760B2 (ja) 画像処理方法およびその、装置、端末並びにコンピュータプログラム
WO2020048308A1 (zh) 多媒体资源分类方法、装置、计算机设备及存储介质
CN111079576B (zh) 活体检测方法、装置、设备及存储介质
WO2020224479A1 (zh) 目标的位置获取方法、装置、计算机设备及存储介质
WO2020221012A1 (zh) 图像特征点的运动信息确定方法、任务执行方法和设备
CN110555839A (zh) 缺陷检测识别方法、装置、计算机设备及存储介质
CN110163380B (zh) 数据分析方法、模型训练方法、装置、设备及存储介质
JP2022542668A (ja) 目標対象物マッチング方法及び装置、電子機器並びに記憶媒体
CN110650379B (zh) 视频摘要生成方法、装置、电子设备及存储介质
CN112581358B (zh) 图像处理模型的训练方法、图像处理方法及装置
WO2020108041A1 (zh) 耳部关键点检测方法、装置及存储介质
CN112150560B (zh) 确定消失点的方法、装置及计算机存储介质
CN110570460A (zh) 目标跟踪方法、装置、计算机设备及计算机可读存储介质
CN110807769B (zh) 图像显示控制方法及装置
CN111339880A (zh) 一种目标检测方法及装置、电子设备和存储介质
CN112257594A (zh) 多媒体数据的显示方法、装置、计算机设备及存储介质
CN111982293B (zh) 体温测量方法、装置、电子设备及存储介质
CN116681755B (zh) 位姿预测方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19873499

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019873499

Country of ref document: EP

Effective date: 20210517