WO2021057536A1 - 一种图像处理方法、装置、计算机设备以及存储介质 - Google Patents

一种图像处理方法、装置、计算机设备以及存储介质 Download PDF

Info

Publication number
WO2021057536A1
WO2021057536A1 PCT/CN2020/115209 CN2020115209W WO2021057536A1 WO 2021057536 A1 WO2021057536 A1 WO 2021057536A1 CN 2020115209 W CN2020115209 W CN 2020115209W WO 2021057536 A1 WO2021057536 A1 WO 2021057536A1
Authority
WO
WIPO (PCT)
Prior art keywords
mask
image
convolution
feature information
target
Prior art date
Application number
PCT/CN2020/115209
Other languages
English (en)
French (fr)
Inventor
胡一凡
郑冶枫
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2021057536A1 publication Critical patent/WO2021057536A1/zh
Priority to US17/499,993 priority Critical patent/US20220028087A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This application relates to the field of computer technology, and in particular to an image processing method, device and related equipment.
  • Targets In the process of image research and application, people are often interested in some parts of each image. These parts are usually called targets or foregrounds, and they generally correspond to areas with unique properties in the image. In order to identify and analyze the target in the image, it is necessary to separate these areas, and on this basis, it is possible to further process the target. Image segmentation is to divide the image into regions with various characteristics and extract the target of interest.
  • the segmentation of the target in the image is through manual segmentation, that is, the position of the target in the image is manually determined, and the image area where the target is located is manually extracted, and then the manually segmented image area is followed for image understanding, etc. .
  • the target in the artificial segmentation image needs to go through the process of artificial position discrimination, artificial target segmentation, etc., which will consume a lot of time, resulting in low efficiency of image segmentation.
  • the embodiments of the present application provide an image processing method, device, and related equipment, which can improve the efficiency of image segmentation.
  • One aspect of the embodiments of the present application provides an image processing method, which is executed by a computer device, and includes:
  • the image segmentation model includes a first unit model and a second unit model
  • the target image area of the target object in the target image is determined according to the updated original mask.
  • an image processing device including:
  • An image acquisition module for acquiring a target image containing the target object
  • the model acquisition module is used to acquire the original mask and the image segmentation model;
  • the image segmentation model includes a first unit model and a second unit model;
  • a pooling module for down-sampling the original mask based on the pooling layer in the first unit model to obtain a down-sampling mask
  • a convolution module configured to extract regional convolution feature information of the target image based on the convolution pooling layer in the second unit model and the down-sampling mask;
  • An update module configured to update the original mask according to the regional convolution feature information
  • the mask determination module is configured to determine the target image area of the target object in the target image according to the updated original mask when the updated original mask satisfies the error convergence condition.
  • Another aspect of the embodiments of the present application provides an electronic medical device, including a biological tissue image collector and a biological tissue image analyzer;
  • the biological tissue image collector acquires a biological tissue image containing a focus object, and acquires an original mask and an image segmentation model;
  • the image segmentation model includes a first unit model and a second unit model;
  • the biological tissue image analyzer down-sampling the original mask based on the pooling layer in the first unit model to obtain a down-sampling mask
  • the biological tissue image analyzer extracts the regional convolution feature information of the biological tissue image based on the convolution pooling layer in the second unit model and the down-sampling mask, and according to the regional convolution feature information Update the original mask;
  • the biological tissue image analyzer determines the lesion image area of the lesion object in the biological tissue image according to the updated original mask.
  • Another aspect of the embodiments of the present application provides a computer storage medium, the computer storage medium stores a computer program, the computer program includes program instructions, and when executed by a processor, the program instructions execute the above-mentioned implementation of the application.
  • the image processing method in the example is a computer storage medium, the computer storage medium stores a computer program, the computer program includes program instructions, and when executed by a processor, the program instructions execute the above-mentioned implementation of the application.
  • FIG. 1 is an image processing system architecture diagram provided by an embodiment of the present application
  • FIGS. 2a-2b are schematic diagrams of an image processing scene provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of an image processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an average pooling provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of maximum pooling provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a second convolutional pooling unit provided by an embodiment of the present application.
  • FIG. 7 is a model architecture diagram of an image processing method provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • Fig. 9a is a functional block diagram of image processing provided by an embodiment of the present application.
  • Figure 9b is a schematic structural diagram of an electronic medical device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • FIG. 1 is a system architecture diagram of image processing provided by an embodiment of the present application.
  • This application relates to a background server 10d and a terminal device cluster.
  • the terminal device cluster may include: a terminal device 10a, a terminal device 10b, ..., a terminal device 10c, and so on.
  • the terminal device 10a obtains a target image containing a target object, where the target object is the object to be segmented.
  • the terminal device 10a sends the above-mentioned target image to the backend server 10d.
  • the backend server 10d obtains the original mask and the trained image segmentation model.
  • the original mask is down-sampled to obtain the down-sampling mask.
  • the regional feature information of the target image is extracted, the original mask is updated according to the regional feature information, and the original mask is updated iteratively.
  • the background server 10d detects that the updated original mask satisfies the error convergence condition, it determines the image area of the target object in the target image according to the updated original mask.
  • the background server 10d can deliver the determined image area to the terminal device 10a, and the terminal device 10a can display the target image and mark the image area where the target object is located on the target image; or the terminal device 10a can directly output the target object in the target image.
  • the image area in the image can be
  • the terminal device 10a may also obtain the image segmentation model and the original mask, and then the terminal device 10a determines the image area of the target object in the target image.
  • the terminal device 10a, terminal device 10b, terminal device 10c, etc. shown in FIG. 1 may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile internet device (MID, mobile internet device), a wearable device, and the like.
  • MID mobile internet device
  • FIGS. 2a-2b take as an example how the terminal device 10a determines the area of the target object in the target image according to the original mask and the image segmentation model:
  • FIGS. 2a-2b are schematic diagrams of an image processing scene provided by an embodiment of the present application.
  • the terminal device 10a obtains the image 20a to be segmented, wherein the image 20a contains the object to be segmented-a puppy, that is, the purpose of image segmentation is to segment the puppy in the image 20a from the image 20a.
  • the terminal device 10a obtains the original mask 20b with the same size as the image 20a.
  • the original mask 20b and the image 20a have the same size, which means that each unit mask in the original mask 20b and each pixel in the image 20a have one-to-one.
  • the value of all unit masks in the original mask 20b may be 0.
  • the original mask 20b includes 16 unit masks (4 ⁇ 4), and the size of the corresponding image 20a is also 4 ⁇ 4.
  • the terminal device 10a obtains the image segmentation model 20c.
  • the image segmentation model 20c includes a first unit model and a second unit model.
  • the first unit model contains 1 pooling layer 1, that is, only the pooling operation is performed in the first unit model;
  • the second unit model contains 2 convolutional layers and 2 pooling layers, which means that In the second unit model, convolution and pooling operations are performed.
  • the number of pooling layers included in the first unit model and the number of convolutional layers and pooling layers included in the second unit model are not limited.
  • the terminal device 10a inputs the original mask 20b into the first unit model, and performs a pooling operation on the original mask 20b through the pooling layer 1 in the first unit model.
  • the pooling operation is to reduce the size of the original mask 20b and pool Operations can include average pooling operations and maximum pooling operations.
  • the original mask 20b with a size of 4 ⁇ 4 undergoes the pooling operation of the pooling layer to obtain a mask 20f, and the size of the mask 20f is reduced to 2 ⁇ 2.
  • the terminal device 10a inputs the image 20a into the second unit model, and performs convolution and pooling operations on the image 20a through the convolution layer 1 and the pooling layer 1 in the second unit model, and extracts the convolution feature information 20d of the image 20a ,
  • the size of the feature map of the convolution feature information 20d is 2 ⁇ 2, and the number of channels is 4.
  • the convolutional layer 2 and pooling layer 2 in the second unit model and the pooled mask 20f convolution and pooling are performed on the convolution feature information 20d to extract the deeper convolution features of the image 20a Information 20e, the feature map size of the deeper convolution feature information 20e is also 2 ⁇ 2, and the number of channels is still 4, that is, the size of the input data and the size of the output data of the convolutional layer 2 and the pooling layer 2 are the same.
  • the mask 20f is input to the pooling layer to obtain a smaller size mask; if the second unit model also includes convolutional layer 3, pooling layer 3, and convolution Layer 4 and pooling layer 4, and then based on convolutional layer 3, pooling layer 3, convolutional layer 4, and pooling layer 4, as well as smaller-sized masks to extract deeper convolution feature information.
  • the last extracted convolution feature information is subjected to a deconvolution operation, so that the size of the deconvolution feature map is the same as the size of the image 20a.
  • the deconvoluted feature information is mapped to a mask 20g, and the size of the mask 20g and the image 20a are the same.
  • the new original mask is input into the first unit model, and the image 20a is similarly input into the second unit model, and the original mask is updated again in the above-mentioned manner, and iteratively updated continuously.
  • the difference amount between the original mask update and before the update is less than the difference amount threshold, the original mask updated last time is used as the target mask 20g.
  • the value of the unit mask in the target mask 20g is either 1 or 0. If the value of the unit mask is 0, it means that the pixel corresponding to the unit mask is the pixel of the background; if the unit is The value of the mask is 1, indicating that the pixel corresponding to the unit mask is the pixel where the puppy is located.
  • the target mask 20g there are 6 unit masks with a value of 1 (as shown in FIG. 2g, the unit mask with a value of 1 is marked with a thick line), and the values of the remaining unit masks are all 0.
  • the corresponding area of the unit mask with a value of 1 in the target mask 20g in the image 20a is the image area where the puppy is located.
  • the terminal device 10a may use a dotted line to identify the image area where the puppy is located to obtain the image 20h; or the terminal device may directly extract the unit image where the puppy is located from the image 20a.
  • the user terminal 10a may play a preset animation on the screen during the process of determining the area where the puppy is in the image 20a.
  • stop playing the animation and display the image 20h of the image area where the puppy is located on the screen; or directly display the unit image where the puppy is located on the screen.
  • the region convolution feature information of the target image (such as the image 20a in the above-mentioned embodiment) is extracted, and the original mask (such as the original in the above-mentioned embodiment) is updated.
  • the mask 20b refer to the following embodiments corresponding to FIGS. 3 to 9.
  • FIG. 3 is a schematic diagram of an image processing method provided by an embodiment of the present application, which can be executed by the computer device shown in FIG. 11.
  • the image processing method may include the following steps:
  • Step S101 Obtain a target image containing a target object, and obtain an original mask and an image segmentation model; the image segmentation model includes a first unit model and a second unit model.
  • the terminal device 10a acquires the target object comprises (FIG. 2a- 2b as described above corresponding to FIG embodiment dog embodiment) of the target image I S (FIG. 2a- 2b as described in FIG embodiment corresponding to the image 20a embodiment, the target image I s size can be expressed as A1 ⁇ B1), wherein the target object is a target image I s of the object to be segmented.
  • the target object can be a lesion object
  • the corresponding target image containing the lesion object can be a biological tissue image
  • the biological tissue image can be a CT (Computed Tomography) image, MRI (Magnetic Resonance Imaging, magnetic resonance imaging)
  • CT Computer Tomography
  • MRI Magnetic Resonance Imaging, magnetic resonance imaging
  • the image is either an endoscopic image or the like, and the target image can be a two-dimensional image or a three-dimensional image.
  • the terminal device 10a according to the size of the target image I s (A1 ⁇ B1) generates an original mask res mask having the same size as the target image (FIG. 2a- 2b as described in FIG. 20b corresponding original mask in the embodiment, the original cover
  • the size of the film mask res can also be expressed as A1 ⁇ B1)
  • the target image I s (A1 ⁇ B1) is a combination of multiple pixels
  • the original mask mask res (A1 ⁇ B1) is the original mask from multiple units Composition, and there is a one-to-one correspondence between the pixel points and the unit original mask.
  • the value of all units of the original mask in the original mask mask res can be 0 (in this application, the value 0 is the first value), of course, it can also be 1 (the value 1 is the second number) or random number.
  • the terminal device acquires an image segmentation model (such as the image segmentation model 20c in the corresponding embodiment in Figure 2a- Figure 2b above), where the image segmentation model includes a first unit model and a second unit model, and the first unit model includes only the pooling layer (The number of pooling layers can be one or more).
  • the target object I s does not have a fixed shape and size in the target image, it is necessary to preprocess the segmented image based on the preprocessing rules, and then input the image segmentation model. After preprocessing, the accuracy of subsequent segmentation can be improved.
  • the following is a specific description of the preprocessing of the image to be segmented:
  • X nor is any pixel after normalization X res represents the pixel value of any pixel of the original image
  • X max represents the maximum pixel value in the original image
  • X min represents the minimum pixel value in the original image.
  • the normalized original image can be further translated, rotated, or symmetrical, all for the purpose of positioning the target object at the center of the image.
  • the terminal device can also obtain the size threshold, and scale the image according to the size threshold, so that the scaled image is the same as the size threshold.
  • the above-mentioned normalization of the original image, or translation, rotation, symmetry, or scaling can be called preprocessing the original image.
  • the original image can be preprocessed according to the actual situation of the original image, and the terminal device will perform the corresponding preprocessing on the original image. after pretreatment resulting image, it is called a target image I s.
  • Step S102 based on the pooling layer in the first unit model, down-sampling the original mask to obtain a down-sampling mask.
  • the terminal device inputs the original mask mask res (A1 ⁇ B1) into the first unit model, and down-samples the original mask mask res (A1 ⁇ B1) based on the pooling layer in the first unit model to obtain the following Sampling mask mask input , where average pooling can be used to downsample the original mask, or maximum pooling can be used to downsample the original mask.
  • the size of the down-sampling mask is smaller than the size of the original mask, but the number of channels of the down-sampling mask and the number of channels of the original mask are the same.
  • the original mask is continuously down-sampled based on multiple pooling layers, that is, the size of the mask after the down-sampling is continuously reduced, and the terminal device can
  • the multiple masks obtained after multiple downsampling are called downsampling masks.
  • the size of the down-sampling mask mask input1 can be expressed as (A1/2,B1/2); if another pooling layer then downsamples the downsampling mask mask input1 again, the size of the downsampling mask mask input2 can be expressed as (A1/4,B1/4), Both the above-mentioned down-sampling mask mask input1 and down-sampling mask mask input2 can be referred to as down-sampling masks.
  • Average pooling means that in a unit data set, the average value of the data contained in the unit data set is used as the representative of the unit data set; maximum pooling means that in the unit data set, the largest data in the unit data set is taken as the unit data Representative of the set.
  • FIG. 4 it is a schematic diagram of an average pooling provided by an embodiment of the present application.
  • the following takes the pooling window across the upper left corner of the input data as an example: at this time, the pooling window contains value 1, value 1, value 5, and value 1.
  • Calculate the average value of the above 4 values, and get (1+1+5+1)/4 2, so after the pooling window crosses the upper left corner of the input data, the average pooling will get the value 2, and the rest of the input data
  • the above methods can be used to perform average pooling respectively to obtain output data.
  • FIG. 5 it is a schematic diagram of maximum pooling provided by an embodiment of the present application.
  • a 2 ⁇ 2 pooling window is used to cross the input data with a step size of 2, and the maximum value in each window is calculated.
  • the following takes the pooling window across the upper left corner of the input data as an example: at this time, the pooling window contains value 1, value 1, value 5, and value 1.
  • the maximum value of the above four values is 5, so after the pooling window crosses the upper left corner of the input data, the value 5 is obtained after the maximum pooling, and the rest of the input data can be pooled separately in the above manner to obtain the output data .
  • Step S103 based on the convolution pooling layer in the second unit model and the down-sampling mask, extract the regional convolution feature information of the target image, and update the original mask according to the regional convolution feature information. membrane.
  • the terminal device inputs the target image I s (A1 ⁇ B1) into the second unit model.
  • the second unit model may include one or more convolutional pooling layers.
  • the second unit model includes one or more convolutional pooling layers. Each convolutional pooling layer is explained separately. First, how to update the original mask when only one convolutional pooling layer is included in the second unit model:
  • the corresponding first unit model also contains only one pooling layer (so the number of downsampling mask input is only 1), and the first volume
  • the pooling layer includes a first convolutional pooling unit (as shown in the above-mentioned Figure 2a-2b corresponding to the convolutional layer 1 + pooling layer 1 in the embodiment) and N second convolutional pooling units (as shown in the above figure 2a- Figure 2b correspond to the convolutional layer 2+pooling layer 2) in the embodiment, and N is an integer greater than 0.
  • the following takes the first convolutional pooling layer including a second convolutional pooling unit as an example Be explained.
  • the target image I s (A1 ⁇ B1) for convolution operation and pooled to obtain characteristic information input convolution T1 (convolution of the input feature information
  • the size of T1 can be expressed as A2 ⁇ B2, C2), and the input convolution feature information T1 can be regarded as a combination of feature maps of C2 channels.
  • the feature map size of each channel is A2 ⁇ B2.
  • the feature map size A2 ⁇ B2 of each channel is equal to the size of the down-sampling mask mask input .
  • the input convolution feature information T1 (A2 ⁇ B2, C2) is encoded (that is, the convolution operation) to generate the second convolution feature information T2 (A2 ⁇ B2 , C2), where the size of the second convolution feature information T2 is the same as the size of the input convolution feature information T1, where the same refers to the same number of channels and the same feature map size.
  • I x in formula (2) represents the feature map of the x-th channel of the second convolution feature information T2 (A2 ⁇ B2, C2), and C2 represents the number of channels of the second convolution feature information.
  • k 11 ,k 21 ,k 12 ,k 22 ,k 13 ,k 23 ...k 1x ,k 2x can be obtained, and the above k 11 ,k 21 ,k 12 ,k 22 ,k 13 ,k 23 ...k 1x ,k 2x as the pooling vector.
  • the second convolution feature information includes 3 channels of feature maps with a size of a ⁇ b
  • the pooling vector can be obtained: [k 11 ,k 21 ,k 12 ,k 22 ,k 13 ,k 23 ].
  • the down-sampling mask mask input (A2 ⁇ B2) is the same for all channels.
  • the pooling vector is converted into a target vector: [k' 11 ,k' 21 ,k' 12 ,k' 22 ,k' 13 ,k' 23 ... k'1x ,k' 2x ], where the dimensions of the target vector and the pooling vector are the same, for example, if the dimension of the pooling vector is 1 ⁇ 1 ⁇ 2C2, then the dimension of the target vector is also 1 ⁇ 1 ⁇ 2C2.
  • K x k' 1x *mask input +k' 2x *(1-mask input ),x ⁇ 1,2,...,C (3)
  • the size of A2 ⁇ B2, C2) the size of the area convolution feature information T reg (A2 ⁇ B2, C2).
  • the result obtained is auxiliary convolution feature information. Then use the auxiliary convolution feature information as the new input convolution feature information, and input the new input convolution feature information into the next second convolution pooling unit, and keep looping until the last second convolution pooling unit is output
  • the feature information of is used as the regional convolution feature information T reg of the target image.
  • FIG. 6 is a schematic diagram of a second convolutional pooling unit provided by an embodiment of the present application.
  • ROI-SE Block dependent convolution unit
  • the data size of the input dependent convolution unit is W ⁇ H ⁇ C, that is, the number of channels is C, and the feature map size is W ⁇ H.
  • the input data is convolved based on the convolution function (Convolution) in the dependent convolution unit.
  • the convolution operation here does not change the size of the input data, and the size of the output data after the convolution operation is still W ⁇ H ⁇ C.
  • the pooling vector is composed of two vectors of size 1 ⁇ 1 ⁇ C, and these two vectors are the vectors obtained by pooling the downsampling mask, and After convolution, the output data of size W ⁇ H ⁇ C is a vector obtained by unmasked pooling.
  • the pooling vector with a size of 1 ⁇ 1 ⁇ 2C is nonlinearly converted into an auxiliary vector, where the size of the auxiliary vector can be 1 ⁇ 1 ⁇ 2C/r ;
  • the auxiliary vector with a size of 1 ⁇ 1 ⁇ 2C/r is nonlinearly converted into a target vector, where the size of the target vector can be 1 ⁇ 1 ⁇ 2C.
  • the target vector can be regarded as a vector composed of two vectors, and the size of the above two vectors is 1 ⁇ 1 ⁇ C.
  • the output data that depends on the convolution unit is obtained, and the size of the output data is also W ⁇ H ⁇ C, where the calculation process of formula (3) can correspond to the figure Mask scale conversion, non-mask scale conversion and scale conversion in 6.
  • the terminal device deconvolves the regional convolution feature information T reg output by the convolution pooling layer to obtain target convolution feature information T out (the size may be A1 ⁇ B1, T).
  • Deconvolution of the convolution object is to increase the area of the feature information of FIG feature size, feature information such that the target convolution T out after deconvolution FIG feature size to the target image I s (A1 ⁇ B1) of the same size.
  • the terminal device fully connects the target convolution feature information T out .
  • the full connection is to calculate the average value of the feature maps of multiple channels at the same position.
  • the output size is A1 ⁇ B1 mask, which is called the mask to be updated.
  • the size of the updated mask is the same as the size of the original mask.
  • the terminal device can directly use the mask to be updated as the new original mask.
  • the terminal device can also adjust the value of the unit to be updated in the mask to be updated that is less than or equal to the mask threshold (the mask threshold can be 0.5) to the first value, where the first value can be 0;
  • the mask to be updated adjust the value of the mask to be updated in units greater than the mask threshold to a second value, where the second value can be 1, so that a binary mask can be obtained.
  • the value in the binary mask is either Either the first value or the second value.
  • the value of the mask to be updated is adjusted to increase the difference between the target image area (the target image area is the image area of the target object in the target image) and the non-target image area.
  • the terminal device then uses the above-mentioned binary mask as the new original mask.
  • the original mask has been updated once, and the above process is performed again, that is, the new original mask is input into the first unit model, and the down-sampling is performed to obtain the new down-sampling mask; the target image is input into the second unit model, According to the new down-sampling mask and the first convolutional pooling unit and N second convolutional pooling units in the second unit model, the new regional convolution feature information of the target image is extracted, and the original mask is continuously updated iteratively. membrane.
  • the second unit model contains multiple convolutional pooling layers
  • the corresponding first unit model also contains multiple pooling layers
  • the multiple convolutional pooling layers in the second unit model can be divided into a first convolutional pooling layer and a second convolutional pooling layer, where the top convolutional pooling layer in the second unit model is the first Convolutional pooling layer, and the remaining convolutional pooling layers are the second convolutional pooling layer; then correspondingly, the original mask includes the first original mask corresponding to the first convolutional pooling layer and the
  • the second original masks corresponding to the second convolutional pooling layers can be understood as the first original mask generated by the first pooling layer in the first unit model, and the second original mask is generated by the first pooling layer in the first unit model. Generated by the second pooling layer in the unit model.
  • the target image and the first original mask are input to the first convolution pooling layer, and the target image is convolved and pooled to generate the first convolution feature information.
  • the pooling layer performs convolution and pooling on the first convolution feature information to obtain the regional convolution feature information of the target image.
  • the second convolution adjacent to the current first convolutional pooling layer will be used
  • the pooling layer is used as the new first convolutional pooling layer
  • the first convolution feature information is used as the target image
  • the second original mask adjacent to the first original mask is used as the new first original mask.
  • the new target image and the new first original mask are input to the new first convolution pooling layer, and the new target image is convolved and pooled to generate new first convolution feature information. At this time, check whether there is only one second convolutional pooling layer.
  • the second convolutional pooling layer inputs the new first convolution feature information and the remaining second original mask into the second convolutional pooling layer.
  • Convolution and pooling are performed on the first convolutional feature information of the target image to obtain the regional convolutional feature information of the target image; if not, the second convolutional pooling layer adjacent to the current first convolutional pooling layer is used as the new The first convolution pooling layer of, takes the newly generated first convolution feature information as the new target image, and takes the second original mask adjacent to the current first original mask as the new first original mask, The loop continues until all the second convolutional pooling layers participate in the operation, and the last second convolutional pooling layer outputs the regional convolution feature information of the target image.
  • the superposition operation here refers to the superposition between the first convolution feature information and the third convolution feature information
  • the third convolution feature information is: 50 ⁇ 50 ⁇ 100
  • the size of the neighboring first convolution feature information is: 50 ⁇ 50 ⁇ 140
  • the size of the superimposed convolution feature information is: 50 ⁇ 50 ⁇ 240.
  • the third convolution feature information and the neighbor first convolution feature information are superimposed and convolved in the dimension of the number of channels.
  • Product operation to obtain reference convolution feature information The obtained reference convolution feature information is used as the new regional convolution feature information, and then the convolution pooling layer adjacent to the convolution pooling layer corresponding to the new regional convolution feature information corresponds to the first convolution feature information corresponding to the convolution pooling layer As the first convolution feature information of the new neighbor.
  • the feature map size of the new third convolution feature information is also equal to the feature of the new neighbor first convolution feature information. Figure size.
  • the first convolution feature information in addition to the neighboring first convolution feature information currently exists, the first convolution feature information has not been superimposed, and the new third convolution feature information and the new neighbor first convolution feature information are in the dimension of the number of channels Perform superposition and convolution operations to obtain new reference convolution feature information.
  • the target convolution feature information is obtained, the target convolution feature information is subjected to a deconvolution operation to obtain the deconvolution feature information.
  • the size of the feature map of the deconvolution feature information is the same as the size of the target image.
  • Full connection can be to calculate the average value of the feature maps of multiple channels at the same position, output the mask to be updated, and then convert the mask to be updated into a binary mask (The value in the binary mask is either the first value or the second value), and then the binary mask is used as a new original mask.
  • each convolutional pooling layer in the second unit model includes a first convolutional pooling unit and N second convolutional pooling units, where N is an integer greater than 0, which is based on each convolution
  • N is an integer greater than 0, which is based on each convolution
  • the specific process of the pooling layer generating the first convolutional feature information and outputting the regional feature information of the target image based on the last convolutional pooling layer at the bottom can refer to the above description, that is, when only one volume is included in the second unit model
  • the convolutional pooling layer is used, how does the convolutional pooling layer determine the regional convolutional feature information?
  • the above is a convolutional pooling layer and directly output the regional convolutional feature information.
  • the second unit model includes multiple convolutional pooling layers . Multiple first convolution feature information will be output in the middle, and the last convolution pooling layer will output regional convolution feature information. Although the input data is different, the calculation process involved in the convolution pooling layer is the same.
  • Step S104 When the updated original mask satisfies the error convergence condition, the target image area of the target object in the target image is determined according to the updated original mask.
  • the essence of updating the original mask each iteration is to generate a new binary mask, use the new binary mask as the new original mask, and then input the second unit model. But in this iterative update process, the first unit model, the second unit model, and the target image are unchanged, and only the original mask is changed.
  • the terminal device obtains the update times threshold. If the number of times to update the original mask reaches the update times threshold, it means that the last updated original mask satisfies the error convergence condition, then the terminal device can use the last updated original mask as the target mask ( As shown in Fig. 2a-Fig. 2b above, corresponding to the target mask 20g in the embodiment).
  • the terminal device calculates the error between the original mask before the update and the original mask after the update, and detects whether the error is less than a preset error threshold. If so, it means that the updated original mask meets the error convergence condition.
  • the terminal device can use the updated original mask as the target mask; if not, it means that the updated original mask has not yet met the error convergence condition, indicating that the original mask needs to be updated iteratively until the updated original mask satisfies Error convergence conditions.
  • the first error l1 can be determined according to the following formula (4):
  • the second error l2 can be determined:
  • the terminal device has determined the target mask. From the foregoing, it can be known that the value of the unit target mask in the target mask is either a first value smaller than the mask threshold or a second value larger than the mask threshold.
  • the size of the target mask is the same as the size of the target image, and each unit mask of the target mask has a one-to-one correspondence with each pixel of the target image.
  • the meaning of the target mask is: if the value of the unit target mask is the first value, it means that the attribute of the pixel corresponding to the unit target mask is the background attribute; if the value of the unit target mask is the second value, it means The attribute of the pixel corresponding to the unit target mask is the attribute of the target object corresponding to the target object.
  • the target mask is: The above target mask contains 4 unit target masks.
  • the target object is a lesion object, it means that the attribute of the first pixel of the target image (the pixel in the upper left corner) is the background attribute, the second pixel and the third pixel
  • the attribute of is the attribute of the lesion object
  • the attribute of the fourth pixel is the background attribute, that is, the first pixel and the fourth pixel are background pixels
  • the second pixel and the fourth pixel are background pixels.
  • the three pixels are the pixels of the lesion object.
  • the category to which each pixel of the target object belongs can be determined through the target mask.
  • the terminal device determines the position information of the unit target mask greater than the mask threshold.
  • the image area corresponding to the position information is regarded as the area of the target object in the target image, which is called the target image area (For example, Figures 2a-2b above correspond to the image area where the puppy is located in the embodiment).
  • the target image is a biological tissue image and the target object is a lesion object
  • the target image area containing the lesion object is the lesion image area
  • the terminal device may mark the target image area in the target image, for example, mark the boundary of the target image area with a dotted line, or highlight the target image area.
  • the number of target objects can be one or more. If the target object is one (belonging to two classification problems), then the number of the corresponding original mask and target mask is only one, and the target Each unit target mask in the mask can indicate that the attribute of the corresponding pixel is either the background attribute or the target object attribute.
  • each unit target mask in each target mask can be represented The probability that the corresponding pixel belongs to the k-th target object attribute.
  • a certain pixel does not belong to any target object attribute, then the pixel belongs to the background attribute.
  • FIG. 7 is a model architecture diagram of an image processing method provided by an embodiment of the present application.
  • the first unit model includes 4 pooling layers
  • the second unit model includes 4 convolutional pooling layers as an example. .
  • the size of the target image is 192*168*128, indicating that the target image is a three-dimensional image.
  • the original mask is an all-zero three-dimensional image with a size of 192*168*128.
  • each convolutional pooling layer includes 1 convolutional pooling unit and 2 dependent convolutional units, which can correspond to the aforementioned 1 first convolutional pooling unit and 2 A second convolution pooling unit) to extract the feature information of the target image, and obtain the first convolution feature information 1 with a size of 96*84*64, 64 and the first volume with a size of 48*42*32, 128, respectively.
  • Convolution feature information 2 the first convolution feature information 3 with a size of 24*21*16,256, and the regional convolution feature information 4 with a size of 12*11*8,512, of which, among the above four convolution feature information,
  • the a*b*c before the comma represents the size of the feature map, and the value after the comma represents the number of channels.
  • the size of the output data of the convolutional pooling unit the size of the output data of the dependent convolutional unit.
  • the deconvolution feature information 3 and the size are The 24*21*16,256 first convolution feature information 3 is superimposed and convolved to obtain superimposed convolution feature information 3 with a size of 24*21*16,512.
  • Deconvolution is performed on the superimposed convolution feature information 3 with a size of 24*21*16,512, and the deconvolution feature information 2 with a size of 48*42*32,512 is obtained.
  • the deconvolution feature information 2 and the size are The first convolution feature information 2 of 48*42*32,128 is superimposed and convolved to obtain superimposed convolution feature information 2 of size 48*42*32,256.
  • the deconvolution feature information 1 and the size are
  • the first convolution feature information 1 of 96*84*64,64 is superimposed and convolved to obtain target convolution feature information of size 96*84*64,128.
  • the target convolution feature information with the size of 96*84*64,128 is deconvolved to obtain the deconvolution feature information with the size of 192*168*128,128.
  • the above-mentioned mask to be updated with a size of 192*168*128 is used as a new original mask and is continuously updated.
  • the area of the target object in the target image can be determined according to the latest original mask.
  • FIG. 8 is a schematic flowchart of an image processing method provided by an embodiment of the present application, which may be executed by the computer device shown in FIG. 11.
  • the image processing method includes the following steps:
  • Step S201 Obtain a sample image including the target object, and obtain a sample original mask and a sample image segmentation model; the sample image segmentation model includes the first unit model and the sample second unit model.
  • the image segmentation model includes the first unit model and the second unit model, and the first unit model only contains the pooling layer, so there are no model variables in the first unit model.
  • the parameters need to be trained, that is, the first unit model does not need to be trained, only the second unit model needs to be trained.
  • the terminal device obtains the sample image containing the target object, obtains the sample original mask and the sample image segmentation model, where the value of each unit sample original mask in the sample original mask may be the first value, that is, the value 0.
  • the sample image segmentation model includes a first unit model and a sample second unit model.
  • Step S202 based on the pooling layer in the first unit model, down-sampling the original sample mask to obtain a sample down-sampling mask.
  • the sample original mask is down-sampled to obtain the sample down-sampling mask.
  • Step S203 Generate a prediction mask based on the second unit model of the sample and the sample down-sampling mask, and determine the sample mask according to the image area of the target object in the sample image.
  • the sample image and the sample down-sampling mask are input into the second unit model of the sample to generate the prediction mask mask pre .
  • the specific process of forward propagation to generate the prediction mask mask pre is the same as the process of generating the target mask described above.
  • Step S204 training the second unit model of the sample according to the sample image, the prediction mask and the sample mask to obtain the image segmentation model.
  • the prediction mask mask pre the sample image I t, and the sample mask mask t are substituted into the following formula (7) to generate intensity loss:
  • backpropagation adjusts the model variable parameters in the second unit model of the sample to obtain the target parameter.
  • the target parameter is the value of the model variable parameter in the second unit model of the sample when the target loss is the smallest.
  • the model variable parameters in the second unit model of the sample are replaced with the target parameters. So far, an iterative update of the model variable parameters of the second unit model of the sample is completed.
  • the sample second unit model at this time is the sample second unit model that has been updated once).
  • the terminal device combines the trained second unit model and the first unit model into an image segmentation model.
  • Step S205 Obtain a target image containing the target object, obtain an original mask and an image segmentation model; the image segmentation model includes a first unit model and a second unit model, based on the pooling layer in the first unit model, The original mask is down-sampled to obtain a down-sampling mask.
  • Step S206 based on the convolution pooling layer in the second unit model and the down-sampling mask, extract the regional convolution feature information of the target image, and update the original mask according to the regional convolution feature information. membrane.
  • Step S207 When the updated original mask satisfies the error convergence condition, the target image area of the target object in the target image is determined according to the updated original mask.
  • step S205 to step S207 For the specific implementation process of step S205 to step S207, refer to step S101 to step S104 in the embodiment corresponding to FIG. 3 above.
  • FIG 9a is a functional module diagram of image processing provided by an embodiment of the present application.
  • the front end A can receive the user input to be segmented Based on image preprocessing rules (for example, image normalization, image translation and/or image rotation, etc.), the medical image is preprocessed to obtain the preprocessed image.
  • image preprocessing rules for example, image normalization, image translation and/or image rotation, etc.
  • the preprocessed image is passed to the backend, and the trained image segmentation model is stored in the backend.
  • the background can segment the recognized area from the medical image to obtain the lesion image.
  • the background sends the lesion image to the front end B (the front end B and the front end A can be the same Front end), Front end B can display the lesion image or further analyze the lesion image.
  • FIG. 9b is a schematic structural diagram of an electronic medical device provided by an embodiment of the present application.
  • the electronic medical device may be the terminal device in the corresponding embodiment in FIGS. 1 to 9a.
  • the electronic medical equipment may include a biological tissue image collector and a biological tissue image analyzer. The above-mentioned electronic medical equipment can collect medical images and analyze the medical images.
  • the specific process includes the following steps:
  • step S301 the biological tissue image collector acquires a biological tissue image containing the lesion object.
  • the biological tissue image collector acquires a biological tissue image containing the lesion object. If the biological tissue image is an MRI image, the biological tissue image collector can be an MRI machine; if the biological tissue image is a CT image, then the biological tissue image collector can be a CT machine; if the biological tissue image is a mammogram, then the biological tissue The image collector can be a molybdenum target machine.
  • Step S302 the biological tissue image analyzer obtains the original mask and the image segmentation model; the image segmentation model includes a first unit model and a second unit model; the biological tissue image analyzer is based on the pooling layer in the first unit model , Performing down-sampling on the original mask to obtain a down-sampling mask.
  • the biological tissue image analyzer obtains a trained image segmentation model, and obtains an original mask with the same size as the biological tissue image.
  • the biological tissue image analyzer inputs the original mask into the first unit model, and performs a pooling operation on the original mask based on the pooling layer in the first unit model to obtain a down-sampling mask, where the pooling operation here can be
  • the maximum pooling operation can also be an average pooling operation.
  • step S102 For the specific process of generating the down-sampling mask by the biological tissue image analyzer based on the first unit model, refer to step S102 in the embodiment corresponding to FIG. 3 above.
  • Step S303 The biological tissue image analyzer extracts the regional convolution feature information of the biological tissue image based on the convolution pooling layer in the second unit model and the down-sampling mask, and according to the regional convolution feature The information updates the original mask.
  • the biological tissue image analyzer inputs the down-sampling mask and the biological tissue image into the second unit model, and extracts the regional convolution feature information of the biological tissue image based on the convolution pooling layer in the second unit model, and according to the regional convolution
  • the feature information generates a mask to be updated, and the mask to be updated is binarized to obtain a binary mask, wherein the size of the mask to be updated is the same as the size of the binary mask and the size of the original mask.
  • the biological tissue image analyzer uses the generated binary mask as a new original mask and re-inputs the first unit model, and iteratively updates the original mask.
  • step S103 For the specific process of updating the original mask based on the second unit model by the biological tissue image analyzer, refer to step S103 in the embodiment corresponding to FIG. 3 above.
  • Step S304 When the updated original mask satisfies the error convergence condition, the biological tissue image analyzer determines the lesion image area of the lesion object in the biological tissue image according to the updated original mask.
  • the biological tissue image analyzer will update the The original mask is used as the target mask.
  • the biological tissue image analyzer determines the position information of the unit target mask greater than the mask threshold.
  • the biological tissue image analyzer regards the image area corresponding to the position information as the focus object in the biological tissue The area in the image is called the lesion image area.
  • the biological tissue image display can identify the lesion image area in the biological tissue image, and display the biological tissue image that identifies the lesion image area.
  • FIG. 10 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • the image processing apparatus 1 can be applied to the terminal equipment in the corresponding embodiments of Figs. 1-9.
  • the image processing apparatus 1 can include: image acquisition module 11, model acquisition module 12, pooling module 13, volume The integration module 14, the update module 15, and the mask determination module 16.
  • the image acquisition module 11 is used to acquire a target image containing a target object
  • the model acquisition module 12 is used to acquire the original mask and the image segmentation model;
  • the image segmentation model includes a first unit model and a second unit model;
  • the pooling module 13 is configured to down-sample the original mask based on the pooling layer in the first unit model to obtain a down-sampling mask
  • the convolution module 14 is configured to extract the regional convolution feature information of the target image based on the convolution pooling layer in the second unit model and the down-sampling mask;
  • the update module 15 is configured to update the original mask according to the regional convolution feature information
  • the mask determining module 16 is configured to determine the target image area of the target object in the target image according to the updated original mask when the updated original mask satisfies the error convergence condition;
  • the image acquisition module 11 is specifically configured to acquire an original image containing the target object, and perform image preprocessing on the original image based on image preprocessing rules to obtain the target image;
  • the image preprocessing rules include image normalization , Image translation, image rotation, image symmetry and image zoom;
  • the mask determination module 16 is specifically configured to determine the updated original mask as the target mask, in the target mask, determine the position information of the unit target mask greater than the mask threshold, and in the target image , Taking an image area corresponding to the position information as the target image area; the target mask includes a plurality of unit target masks.
  • the specific functional implementations of the image acquisition module 11, the model acquisition module 12, the pooling module 13, the convolution module 14, the update module 15, and the mask determination module 16 can refer to the step S101-step in the corresponding embodiment in FIG. 3 above. S104, the details are not repeated here.
  • the convolutional pooling layer in the second unit model includes a first convolutional pooling layer and a second convolutional pooling layer;
  • the downsampling mask includes the same as the first convolutional pooling layer.
  • the convolution module 14 may include: a first convolution unit 141 and a second convolution unit 142.
  • the first convolution unit 141 is configured to perform convolution and pooling on the target image based on the first convolution pooling layer and the first original mask to obtain first convolution feature information;
  • the second convolution unit 142 is configured to perform convolution and pooling on the first convolution feature information based on the second convolution pooling layer and the second original mask to obtain the target image The regional convolution feature information;
  • step S103 For the specific processes of the first convolution unit 141 and the second convolution unit 142, refer to step S103 in the embodiment corresponding to FIG. 3 above.
  • the first convolutional pooling layer includes a first convolutional pooling unit and a second convolutional pooling unit;
  • the first convolution unit 141 is specifically configured to perform convolution and pooling on the target image based on the convolution function and pooling function corresponding to the first convolution pooling unit to obtain input convolution feature information, based on The convolution function corresponding to the second convolution pooling unit encodes the input convolution feature information to generate second convolution feature information, and according to the first original mask, the second convolution
  • the feature maps of the multiple channels of the feature information are pooled separately, a pooling vector is determined, the pooling vector is converted into a target vector based on the activation function corresponding to the second convolutional pooling unit, and the target vector is calculated according to the target vector and
  • the first original mask generates the first convolution feature information; the size of the input convolution feature information, the size of the first convolution feature information, and the size of the second convolution feature information are all the same.
  • step S103 For the specific process of the first convolution unit 141, refer to step S103 in the embodiment corresponding to FIG. 3 above.
  • the update module 15 may include: a deconvolution unit 151, an overlay unit 152, and a determination unit 153.
  • the deconvolution unit 151 is configured to perform deconvolution on the regional convolution feature information to generate third convolution feature information;
  • the superimposing unit 152 is configured to superimpose the third convolution feature information and the first convolution feature information into target convolution feature information, and perform deconvolution and full connection on the target convolution feature information to obtain the Update mask
  • the determining unit 153 is configured to determine the mask to be updated as the original mask; the size of the mask to be updated is the same as the size of the target image;
  • the mask to be updated includes multiple unit masks to be updated
  • the determining unit 153 is specifically configured to adjust the value of the unit to be updated that is less than or equal to the mask threshold value in the to-be-updated mask to a first value, and in the to-be-updated mask, The value of the unit to be updated mask that is larger than the mask threshold is adjusted to a second value to obtain the original mask.
  • step S103 For the specific processes of the deconvolution unit 151, the superimposition unit 152, and the determination unit 153, refer to step S103 in the embodiment corresponding to FIG. 3, which will not be repeated here.
  • the image processing apparatus 1 may include: an image acquisition module 11, a model acquisition module 12, a pooling module 13, a convolution module 14, an update module 15, and a mask determination module 16, and may also include a convergence determination module 17.
  • the convergence determination module 17 is configured to determine that the updated original mask satisfies the error convergence condition if the error between the updated original mask and the original mask before the update is less than the error threshold, or,
  • the convergence determination module 17 is further configured to determine that the updated original mask satisfies the error convergence condition if the number of updates reaches the threshold of the number of updates.
  • step S104 The specific process of the convergence determination module 17 can refer to step S104 in the embodiment corresponding to FIG. 3, which will not be repeated here.
  • the image processing apparatus 1 may include: an image acquisition module 11, a model acquisition module 12, a pooling module 13, a convolution module 14, an update module 15, and a mask determination module 16, and may also include: a sample acquisition module 18 and training module 19.
  • the sample acquisition module 18 is configured to acquire a sample image containing the target object, and acquire a sample original mask and a sample image segmentation model; the sample image segmentation model includes the first unit model and the sample second unit model;
  • the sample acquisition module 18 is further configured to down-sample the original sample mask based on the pooling layer in the first unit model to obtain a sample down-sampling mask;
  • the sample acquisition module 18 is further configured to generate a prediction mask based on the second unit model of the sample and the sample down-sampling mask, and determine the sample mask according to the image area of the target object in the sample image.
  • the training module 19 is configured to train the second unit model of the sample according to the sample image, the prediction mask, and the sample mask to obtain the image segmentation model;
  • the training module 19 is specifically configured to generate a brightness loss based on the sample image, the prediction mask, and the sample mask, generate a segmentation loss based on the prediction mask and the sample mask, and convert the brightness The loss and the segmentation loss are combined as the target loss.
  • the parameter value of the model variable parameter in the sample second unit model is determined, and the sample second unit is updated according to the parameter value
  • the model variable parameters in the model when the number of training times reaches the threshold of the number of training times, the second unit model of the sample after updating the model variable parameters is used as the second unit model, and the first unit model and the second unit model
  • the combination is the image segmentation model.
  • the specific process of the sample acquisition module 18 and the training module 19 can be referred to steps S201 to S204 in the corresponding embodiment of FIG. 8 above, which will not be repeated here.
  • the target object is a lesion object; the target image is a biological tissue image; and the target image area is a lesion image area.
  • This application obtains the target image containing the target object, and obtains the original mask and the image segmentation model.
  • the original mask is down-sampled to obtain the down-sampling mask;
  • the second unit model and the down-sampling mask extract the regional feature information of the target image, update the original mask according to the regional feature information, and iteratively update the original mask until the updated original mask meets the convergence condition, according to
  • the updated original mask can determine the area of the target object in the target image. It can be seen from the above that the region of the target object in the image is automatically segmented in an automated manner. Compared with manual segmentation, the time-consuming of image segmentation can be reduced and the efficiency of image segmentation can be improved.
  • FIG. 11 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the terminal device in the embodiment corresponding to FIG. 1 to FIG. 9 may be a computer device 1000.
  • the computer device 1000 may include: a user interface 1002, a processor 1004, an encoder 1006, and a memory 1008.
  • the signal receiver 1016 is used to receive or send data via the cellular interface 1010, the WIFI interface 1012,..., or the NFC interface 1014.
  • the encoder 1006 encodes the received data into a data format processed by the computer.
  • a computer program is stored in the memory 1008, and the processor 1004 is configured to execute the steps in any one of the foregoing method embodiments through the computer program.
  • the memory 1008 may include volatile memory (e.g., dynamic random access memory DRAM), and may also include non-volatile memory (e.g., one-time programmable read-only memory OTPROM). In some examples, the memory 1008 may further include a memory remotely provided with respect to the processor 1004, and these remote memories may be connected to the computer device 1000 through a network.
  • the user interface 1002 may include a keyboard 1018 and a display 1020.
  • the processor 1004 may be used to call a computer program stored in the memory 1008 to implement:
  • the image segmentation model includes a first unit model and a second unit model
  • the target image area of the target object in the target image is determined according to the updated original mask.
  • the convolutional pooling layer in the second unit model includes a first convolutional pooling layer and a second convolutional pooling layer;
  • the down-sampling mask includes the first convolutional pooling layer and the first convolutional pooling layer.
  • the processor 1004 extracts the regional convolution feature information of the target image based on the convolution pooling layer in the second unit model and the down-sampling mask, it specifically executes the following steps:
  • convolution and pooling are performed on the first convolution feature information to obtain the regional convolution feature information of the target image.
  • the first convolutional pooling layer includes a first convolutional pooling unit and a second convolutional pooling unit;
  • the processor 1004 When the processor 1004 performs convolution and pooling on the target image based on the first convolution pooling layer and the first original mask to obtain first convolution feature information, it specifically executes the following steps:
  • the first convolution feature information is generated; the size of the input convolution feature information, the size of the first convolution feature information, and the second volume
  • the size of the product feature information is the same.
  • processor 1004 when the processor 1004 executes the update of the original mask according to the region convolution feature information, it specifically executes the following steps:
  • the target convolution feature information is deconvolved and fully connected to obtain a mask to be updated, and the mask to be updated is determined as the original mask; the size of the mask to be updated and the target image The dimensions are the same.
  • the mask to be updated includes a plurality of unit masks to be updated
  • the processor 1004 determines that the mask to be updated is the original mask, it specifically executes the following steps:
  • the unit value of the mask to be updated that is greater than the mask threshold is adjusted to a second value to obtain the original mask.
  • the processor 1004 specifically executes the following steps when determining the target image area of the target object in the target image according to the updated original mask:
  • the target mask includes a plurality of unit target masks
  • an image area corresponding to the position information is used as the target image area.
  • the processor 1004 specifically executes the following steps when executing the acquisition of the target image containing the target object:
  • Image preprocessing is performed on the original image based on image preprocessing rules to obtain the target image;
  • the image preprocessing rules include image normalization, image translation, image rotation, image symmetry, and image scaling.
  • processor 1004 further executes the following steps:
  • processor 1004 further executes the following steps:
  • the sample image segmentation model includes the first unit model and the sample second unit model
  • the second unit model of the sample is trained according to the sample image, the prediction mask, and the sample mask to obtain the image segmentation model.
  • the processor 1004 when the processor 1004 executes training the second unit model of the sample according to the sample image, the prediction mask, and the sample mask to obtain the image segmentation model, the processor 1004 specifically executes the following steps:
  • the second unit model of the sample after the model variable parameter is updated is used as the second unit model, and the first unit model and the second unit model are combined into the image segmentation model .
  • the target object is a lesion object; the target image is a biological tissue image; and the target image area is a lesion image area.
  • the computer device 1000 described in the embodiment of the present application can execute the description of the image processing method in the foregoing embodiment corresponding to FIG. 1 to FIG. 9 and may also execute the foregoing description of the image processing method in the foregoing embodiment corresponding to FIG. 10
  • the description of the image processing device 1 will not be repeated here.
  • the description of the beneficial effects of using the same method will not be repeated.
  • a computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor 1004 in the computer device 1000 shown in FIG. 11 reads the computer instruction from a computer-readable storage medium, and the processor 1004 executes the computer instruction, so that the computer device 1000 executes the image described in the above-mentioned various method embodiments. Approach.
  • the embodiments of the present application also provide a computer storage medium, and the computer storage medium stores the aforementioned computer program executed by the image processing apparatus 1, and the computer program includes
  • the program instructions when the processor executes the program instructions, can execute the description of the image processing method in the foregoing embodiments corresponding to FIG. 1 to FIG. 9, and therefore, will not be repeated here.
  • the description of the beneficial effects of using the same method will not be repeated.
  • the program can be stored in a computer readable storage medium, and the program can be stored in a computer readable storage medium. During execution, it may include the procedures of the above-mentioned method embodiments.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

一种图像处理方法、装置、计算机设备以及存储介质，所述方法由计算机设备执行，包括：获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型，基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜，基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜，当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。

Description

一种图像处理方法、装置、计算机设备以及存储介质
本申请要求于2019年9月25日提交中国专利局、申请号为201910912096.1、发明名称为“一种图像处理方法、装置、计算机设备以及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种图像处理方法、装置以及相关设备。
发明背景
在图像的研究和应用过程中，人们往往对各幅图像中的某些部分感兴趣。这些部分通常称为目标或者前景，它们一般对应图像中具有独特性质的区域。为了辨别和分析图像中的目标，需要将这些区域分离出来，在此基础上才有可能对目标进一步地处理。图像分割就是将图像分成各具特性的区域并提取出感兴趣的目标。
目前，对图像中的目标进行分割是通过人工分割，即由人工对图像中的目标的位置进行判别，再手动提取出目标所在的图像区域，后续跟进人工分割出来的图像区域进行图像理解等。
人工分割图像中的目标需要经历人工位置判别，人工目标分割等过程，会耗费大量的时间，造成图像分割的效率低下。
发明内容
本申请实施例提供一种图像处理方法、装置以及相关设备，可以提高图像分割的效率。
本申请实施例一方面提供了一种图像处理方法，由计算机设备执行，包括：
获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜；
当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
本申请实施例另一方面提供了一种图像处理装置，包括：
图像获取模块，用于获取包含目标对象的目标图像；
模型获取模块，用于获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
池化模块，用于基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
卷积模块，用于基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息；
更新模块，用于根据所述区域卷积特征信息更新所述原始掩膜；
掩膜确定模块，用于当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
本申请实施例另一方面提供了一种电子医疗设备，包括生物组织图像采集器和生物组织图像分析器；
所述生物组织图像采集器获取包含病灶对象的生物组织图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
所述生物组织图像分析器基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
所述生物组织图像分析器基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述生物组织图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜；
当更新后的原始掩膜满足误差收敛条件时，所述生物组织图像分析器根据更新后的原始掩膜确定所述病灶对象在所述生物组织图像中的病灶图像区域。
本申请实施例另一方面提供了一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如本申请上述实施例中的图像处理方法。
本申请实施例另一方面提供了一种计算机存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时，执行如本申请上述实施例中的图像处理方法。
附图简要说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种图像处理的系统架构图；
图2a-图2b是本申请实施例提供的一种图像处理的场景示意图；
图3是本申请实施例提供的一种图像处理方法的示意图；
图4是本申请实施例提供的一种平均池化的示意图；
图5是本申请实施例提供的一种最大池化的示意图；
图6是本申请实施例提供的一种第二卷积池化单元的示意图；
图7是本申请实施例提供的一种图像处理方法的模型架构图；
图8是本申请实施例提供的一种图像处理方法的流程示意图；
图9a是本申请实施例提供的一种图像处理的功能模块图；
图9b是本申请实施例提供的一种电子医疗设备的结构示意图；
图10是本申请实施例提供的一种图像处理装置的结构示意图；
图11是本申请实施例提供的一种计算机设备的结构示意图。
实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
请参见图1，是本申请实施例提供的一种图像处理的系统架构图。本申请涉及后台服务器10d以及终端设备集群，终端设备集群可以包括：终端设备10a、终端设备10b、...、终端设备10c等。
以终端设备10a为例，终端设备10a获取包含目标对象的目标图像，其中目标对象即是待分割对象。终端设备10a将上述目标图像发送至后台服务器10d，后台服务器10d接收到目标图像后，获取原始掩膜和已经训练好的图像分割模型。基于图像分割模型中的第一单位模型，对原始掩膜下采样，得到下采样掩膜。基于图像分割模型中的第二单位模型以及下采样掩膜，提取目标图像的区域特征信息，根据区域特征信息更新原始掩膜，不断地迭代更新原始掩膜。当后台服务器10d检测到更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定目标对象在目标图像中的图像区域。
后续，后台服务器10d可以将确定的图像区域下发至终端设备10a，终端设备10a可以显示目标图像，并在目标图像上标记目标对象所在的图像区域；或者终端设备10a可以直接输出目标对象在目标图像中的图像区域。
当然，也可以由终端设备10a获取图像分割模型和原始掩膜，进而由终端设备10a确定目标对象在目标图像中的图像区域。
其中，图1所示的终端设备10a、终端设备10b、终端设备10c等可以是手机、平板电脑、 笔记本电脑、掌上电脑、移动互联网设备(MID，mobile internet device)、可穿戴设备等。
下述图2a-图2b以终端设备10a如何根据原始掩膜和图像分割模型确定目标对象在目标图像中的区域为例进行具体的说明：
请参见图2a-图2b，是本申请实施例提供的一种图像处理的场景示意图。如图2a所示，终端设备10a获取待分割的图像20a，其中图像20a中包含待分割对象-小狗，即图像分割的目的是将图像20a中的小狗从图像20a中分割出来。
终端设备10a获取与图像20a具有相同尺寸的原始掩膜20b，原始掩膜20b与图像20a具有相同尺寸表示原始掩膜20b中的每个单位掩膜与图像20a中的每个像素点具有一一对应关系，初始情况下，原始掩膜20b中所有单位掩膜的取值可以都为0。
从图2a可以看出，原始掩膜20b包含16个单位掩膜(4×4)，对应地图像20a的尺寸也是4×4。
终端设备10a获取图像分割模型20c，从图2a可以看出，图像分割模型20c包含第一单位模型和第二单位模型。
其中，第一单位模型包含1个池化层1，也即是在第一单位模型中只执行池化运算；第二单位模型包含2个卷积层和2个池化层，也即是在第二单位模型中会执行卷积运算和池化运算。当然，第一单位模型所包含的池化层的数量，以及第二单位模型所包含的卷积层和池化层的数量均没有限制。
终端设备10a将原始掩膜20b输入第一单位模型，通过第一单位模型中的池化层1，对原始掩膜20b进行池化运算，池化运算就是降低原始掩膜20b的尺寸，池化运算可以包括平均池化运算和最大池化运算。如图2a所示，尺寸为4×4的原始掩膜20b经过池化层的池化运算后，得到掩膜20f，且掩膜20f的尺寸降低为2×2。
终端设备10a将图像20a输入第二单位模型，通过第二单位模型中的卷积层1和池化层1，对图像20a进行卷积运算和池化运算，提取图像20a的卷积特征信息20d，其中卷积特征信息20d的特征图尺寸为2×2，通道数为4。通过第二单位模型中的卷积层2和池化层2以及池化后的掩膜20f，再对卷积特征信息20d进行卷积运算和池化运算，提取图像20a更深层次的卷积特征信息20e，更深层次的卷积特征信息20e的特征图尺寸也为2×2，通道数仍为4，即卷积层2和池化层2的输入数据的尺寸和输出数据的尺寸相同。
若第一单位模型还包括池化层2，再将掩膜20f输入池化层，以得到尺寸更小的掩膜；若第二单位模型还包括卷积层3、池化层3以及卷积层4和池化层4，再根据卷积层3、池化层3、卷积层4和池化层4，以及尺寸更小的掩膜，提取更深层次的卷积特征信息。
最后通过转置卷积层，将最后一次提取出来的卷积特征信息进行反卷积运算，使得反卷积后的特征图尺寸与图像20a的尺寸相同。通过全连接层，将反卷积后的特征信息映射为掩膜20g，且掩膜20g与图像20a的尺寸相同。
再将掩膜20g中大于掩膜阈值的单位掩膜的取值调整为1，将掩膜20g中小于或等于掩膜阈 值的单位掩膜的取值调整为0，可以知道调整后得到二值掩膜，将二值掩膜作为新的原始掩膜。
至此，就完成了对原始掩膜20b的一次更新。
再将新的原始掩膜输入第一单位模型，同样地将图像20a输入第二单位模型，采用上述方式再次更新原始掩膜，不断地迭代更新。当原始掩膜更新后与更新前之间的差异量小于差异量阈值时，将最后一次更新的原始掩膜作为目标掩膜20g。
可以知道，目标掩膜20g中的单位掩膜的取值要么是1要么是0，若单位掩膜的取值是0，说明该单位掩膜对应的像素点是背景所在的像素点；若单位掩膜的取值是1，说明该单位掩膜对应的像素点是小狗所在的像素点。
在目标掩膜20g中，取值为1的单位掩膜有6个(如图2g所示，取值为1的单位掩膜用粗线条标识)，其余的单位掩膜的取值都为0。
由于单位掩膜与图像20a的像素点具有一一对应关系，因此目标掩膜20g中取值为1的单位掩膜在图像20a中的对应的区域即是小狗所在的图像区域。在图像20a中，终端设备10a可以用虚线将小狗所在的图像区域标识出来，得到图像20h；或者终端设备也可以直接从图像20a中提取出小狗所在的单位图像。
如图2b所示，用户终端10a在确定图像20a中小狗所在区域的过程中，可以在屏幕上播放预设动画。当检测到区域确定完毕时，停止播放动画，将小狗所在的图像区域标识后的图像20h显示在屏幕上；或者直接将提取出小狗所在的单位图像显示在屏幕上。
其中，基于图像分割模型(如上述实施例中的图像分割模型20c)提取目标图像(如上述实施例中的图像20a)的区域卷积特征信息，更新原始掩膜(如上述实施例中的原始掩膜20b)的具体过程可以参见下述图3-图9对应的实施例。
请参见图3，是本申请实施例提供的一种图像处理方法的示意图，可由图11所示的计算机设备执行，如图3所示，图像处理方法可以包括如下步骤：
步骤S101，获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型。
具体的，终端设备10a获取包含目标对象(如上述图2a-图2b对应实施例中的小狗)的目标图像I s(如上述图2a-图2b对应实施例中的图像20a，目标图像I s的尺寸可以表示为A1×B1)，其中目标对象是目标图像I s中待分割的对象。
目标对象可以是病灶对象，那么对应的包含病灶对象的目标图像可以是生物组织图像，生物组织图像可以是CT(Computed Tomography，即电子计算机断层扫描)图像，MRI(Magnetic Resonance Imaging，磁共振成像)图像或者是内窥镜图像等，且目标图像可以是二维图像，也可以是三维图像等。
终端设备10a根据目标图像I s(A1×B1)的尺寸，生成与该目标图像具有相同尺寸的原始掩膜mask res(如上述图2a-图2b对应实施例中的原始掩膜20b，原始掩膜mask res尺寸也可以表示为A1×B1)，目标图像I s(A1×B1)是由多个像素点组合，类似地原始掩膜mask res(A1×B1)是 由多个单位原始掩膜组成，且像素点与单位原始掩膜之间存在一一对应关系。
初始情况下，原始掩膜mask res(A1×B1)中的所有单位原始掩膜的取值可以都为0(在本申请中数值0是第一数值)，当然也可以取值为1(数值1是第二数值)或者随机数。
终端设备获取图像分割模型(如上述图2a-图2b对应实施例中的图像分割模型20c)，其中图像分割模型包括第一单位模型和第二单位模型，第一单位模型中只包括池化层(池化层的数量可以是一个或多个)，第二单位模型中包括卷积池化层(卷积池化层的数量可以是一个或多个)，其中卷积池化层是卷积层和池化层的合称；且第一单位模型中所包含的池化层的数量＝第二单位模型中所包含的卷积池化层的数量。
由于目标对象I s在目标图像中没有固定的形状和大小，因此，有必要基于预处理规则对待分割图像进行预处理后，再输入图像分割模型，预处理后可以提高后续分割的精确度。下面就对待分割图像的预处理进行具体的说明：
获取包含目标对象的原始图像，将原始图像的像素值归一化至0-1区间内，其中可以采用下述公式(1)进行图像归一化：
Figure PCTCN2020115209-appb-000001
其中，X nor是归一化后的任意像素点，X res表示原始图像的任意像素点的像素值，X max表示原始图像中的最大像素值，X min表示原始图像中的最小像素值。
原始图像归一化后，可以对归一化后的原始图像再进行平移、旋转、或者对称，目的都是为了使目标对象位于图像的中心位置。
终端设备还可以获取尺寸阈值，根据尺寸阈值缩放图像，使得尺寸缩放后的图像与尺寸阈值相同。
上述对原始图像进行归一化，或者平移或者旋转或者对称或者缩放都可以称为对原始图像进行预处理，可以根据原始图像的实际情况，对原始图像执行相应的预处理，终端设备将原始图像预处理后所得到的图像，称为目标图像I s。
步骤S102，基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜。
具体的，终端设备将原始掩膜mask res(A1×B1)输入第一单位模型，基于第一单位模型中的池化层，对原始掩膜mask res(A1×B1)进行下采样，得到下采样掩膜mask input，其中可以采用平均池化对原始掩膜下采样，也可以采用最大池化对原始掩膜进行下采样。下采样掩膜的尺寸是小于原始掩膜的尺寸的，但下采样掩膜的通道数和原始掩膜的通道数都是相同的。
若第一单位模型中包含的池化层的数量为多个，基于多个池化层不断地对原始掩膜进行下采样，也即是不断地降低下采样后掩膜的尺寸，终端设备可以将多次下采样后得到的多个掩膜都称为下采样掩膜。
例如，若第一单位模型中的池化层是对半下采样，那么一个池化层对原始掩膜mask res(A1×B1)下采样后，得到下采样掩膜mask input1的尺寸可以表示为(A1/2,B1/2)；若再一个池化层再对下采样掩膜mask input1再次下采样，得到下采样掩膜mask input2的尺寸可以表示为(A1/4,B1/4)，上述下采样掩膜mask input1以及下采样掩膜mask input2都可以称为下采样掩膜。
平均池化是指在单位数据集中，将单位数据集中所包含的数据的平均值作为该单位数据集的代表；最大池化是指在单位数据集中，将单位数据集中的最大数据作为该单位数据集的代表。
如图4所示，是本申请实施例提供的一种平均池化的示意图。以2×2的池化窗口，步长为2划过输入数据，计算每个窗口内的平均值。下述以池化窗口划过输入数据的左上角为例：此时池化窗口中包含数值1、数值1、数值5以及数值1。计算上述4个数值的平均值，得到(1+1+5+1)/4＝2，因此池化窗口划过输入数据的左上角后，平均池化后得到数值2，输入数据的其余部分都可采用上述方式分别进行平均池化，得到输出数据。
如图5所示，是本申请实施例提供的一种最大池化的示意图。同样以2×2的池化窗口，步长为2划过输入数据，计算每个窗口内的最大值。下述以池化窗口划过输入数据的左上角为例：此时池化窗口中包含数值1、数值1、数值5以及数值1。上述4个数值的最大值为5，因此池化窗口划过输入数据的左上角后，最大池化后得到数值5，输入数据的其余部分都可采用上述方式分别进行最大池化，得到输出数据。
步骤S103，基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜。
具体的，终端设备将目标图像I s(A1×B1)输入第二单位模型，第二单位模型中可以包含一个或多个卷积池化层，下面对第二单位模型中包括一个或多个卷积池化层分别进行说明，首先说明第二单位模型中只包括一个卷积池化层时如何更新原始掩膜：
若第二单位模型中只包含一个卷积池化层，那么对应的第一单位模型中也只包含一个池化层(因此下采样掩膜mask input的数量也只有1个)，且第一卷积池化层包括一个第一卷积池化单元(如上述图2a-图2b对应实施例中的卷积层1+池化层1)和N个第二卷积池化单元(如上述图2a-图2b对应实施例中的卷积层2+池化层2)，N是大于0的整数，简单起见下述以第一卷积池化层包括一个第二卷积池化单元为例进行说明。
基于第一卷积池化单元中的卷积函数和池化函数，对目标图像I s(A1×B1)进行卷积运算和池化运算，得到输入卷积特征信息T1(输入卷积特征信息T1的尺寸可以表示为A2×B2,C2)，输入卷积特征信息T1可以看做是C2个通道的特征图组合而成的特征信息，其中每个通道的特征图尺寸是A2×B2，每个通道的特征图尺寸A2×B2等于下采样掩膜mask input的尺寸。
基于第二卷积池化单元中的卷积函数，对输入卷积特征信息T1(A2×B2,C2)进行编码(即是卷积运算)，生成第二卷积特征信息T2(A2×B2,C2)，其中第二卷积特征信息T2的尺寸和输入卷积特征信息T1的尺寸相同，此处的相同是指通道数相同，且特征图尺寸相同。
将下采样掩膜mask input(A2×B2)，和第二卷积特征信息T2(A2×B2×C2)代入下述公式 (2)，计算池化向量：
Figure PCTCN2020115209-appb-000002
其中，公式(2)中的I x表示第二卷积特征信息T2(A2×B2,C2)的第x通道的特征图，C2表示第二卷积特征信息的通道数。根据公式(2)，可以得到k 11,k 21,k 12,k 22,k 13,k 23...k 1x,k 2x，将上述k 11,k 21,k 12,k 22,k 13,k 23...k 1x,k 2x作为池化向量。
可以知道，池化向量中所包含的数值的数量＝第二卷积特征信息的通道数量*2。
举例来说，若第二卷积特征信息的尺寸为a×b,3，即第二卷积特征信息包含3个通道的尺寸为a×b的特征图，将第二卷积特征信息以及下采样掩膜(下采样掩膜的尺寸＝输入卷积特征信息的特征图尺寸＝第二卷积特征信息的特征图尺寸，因此下采样掩膜的尺寸为a×b)代入公式(2)，可以得到池化向量：[k 11,k 21,k 12,k 22,k 13,k 23]。
也可以理解为下采样掩膜mask input(A2×B2)对于所有通道都是相同的。
基于第二池化单元中的全连接函数和激活函数(激活函数可以是Sigmoid函数、tanh函数或者ReLU函数)，将池化向量转换为目标向量：[k’ 11,k' 21,k’ 12,k' 22,k’ 13,k' 23...k’ 1x,k' 2x]，其中目标向量和池化向量的维数是相同的，例如，若池化向量的维数是1×1×2C2，那么目标向量的维数也是1×1×2C2。
再将下采样掩膜mask input(A2×B2)，和目标向量代入下述公式(3)，计算目标图像的区域卷积特征信息T reg：
K x＝k’ 1x*mask input+k' 2x*(1-mask input),x∈1,2,...,C       (3)
将K 1，K 2,，...，K x组合为区域卷积特征信息T reg。
从上述公式(2)和(3)可以看出，当第二卷积池化单位只有一个时，输入卷积特征信息T1(A2×B2,C2)的尺寸＝第二卷积特征信息T2(A2×B2,C2)的尺寸＝区域卷积特征信息T reg(A2×B2,C2)的尺寸。
若第二卷积池化单元的数量是多个，将下采样掩膜mask input(A2×B2)，和目标向量代入公式(3)后，得到的结果是辅助卷积特征信息。再将辅助卷积特征信息作为新的输入卷积特征信息，将新的输入卷积特征信息输入下一个第二卷积池化单元，不断循环，直至将最后一个第二卷积池化单元输出的特征信息作为目标图像的区域卷积特征信息T reg。
请参见图6，是本申请实施例提供的一种第二卷积池化单元的示意图，第二卷积池化单元也可以称为依赖卷积单元(ROI-SE Block)，依赖卷积单元具有以下特征：输入数据的尺寸＝输出数据的尺寸，此处的尺寸是指特征图的尺寸以及通道数量。
如图6所示，设输入依赖卷积单元的数据尺寸是W×H×C，即通道数是C，特征图尺寸是W×H。基于依赖卷积单元中的卷积函数(Convolution)对输入数据进行卷积运算，此处的 卷积运算不改变输入数据的尺寸，卷积运算后输出数据的尺寸仍旧是W×H×C。
将下采样掩膜和上述卷积运算后尺寸为W×H×C的输出数据，代入上述公式(2)，得到尺寸为1×1×2C池化向量。从公式(2)可以看出，池化向量是由两个尺寸为1×1×C的向量组合而成，且这两个向量是对下采样掩膜进行池化所得到的向量，以及对卷积运算后尺寸为W×H×C的输出数据进行非掩膜池化所得到的向量。
基于依赖卷积单元中的全连接函数(FC)以及激活函数，将尺寸为1×1×2C的池化向量非线性转换为辅助向量，其中辅助向量的尺寸可以为1×1×2C/r；再基于依赖卷积单元中的全连接函数(FC)以及激活函数，将尺寸为1×1×2C/r的辅助向量非线性转换为目标向量，其中目标向量的尺寸可以为1×1×2C。
同样地，可以将目标向量看做是两个向量组合而成的向量，且上述两个向量的尺寸都为1×1×C。
将下采样掩膜和目标向量代入上述公式(3)，得到依赖卷积单元的输出数据，且该输出数据的尺寸也为W×H×C，其中公式(3)的计算过程即可对应图6中的掩膜尺度转换、非掩膜尺度转换以及尺度转换。
终端设备将卷积池化层输出的区域卷积特征信息T reg进行反卷积，得到目标卷积特征信息T out(尺寸可以为A1×B1,T)。反卷积的目的是增大区域卷积特征信息的特征图尺寸，使得反卷积后的目标卷积特征信息T out的特征图尺寸与目标图像I s(A1×B1)的尺寸相同。
终端设备将目标卷积特征信息T out进行全连接，全连接即是计算多个通道的特征图在同一个位置的平均值，输出尺寸为A1×B1掩膜，称为待更新掩膜，待更新掩膜的尺寸和原始掩膜的尺寸相同。
终端设备可以直接将待更新掩膜作为新的原始掩膜。
终端设备也可以在待更新掩膜中，将小于或等于掩膜阈值(掩膜阈值可以为0.5)的单位待更新掩膜的取值调整为第一数值，其中第一数值可以是0；在待更新掩膜中，将大于掩膜阈值的单位待更新掩膜的取值调整为第二数值，其中第二数值可以是1，这样可以得到二值掩膜，二值掩膜中的数值要么是第一数值要么是第二数值，调整待更新掩膜的数值是为了增大目标图像区域(目标图像区域是目标对象在目标图像中的图像区域)和非目标图像区域之间的差异性。终端设备再将上述二值掩膜作为新的原始掩膜。
至此，完成了对原始掩膜的一次更新，再执行上述过程，即将新的原始掩膜输入第一单位模型，进行下采样，得到新的下采样掩膜；将目标图像输入第二单位模型，根据新的下采样掩膜和第二单位模型中的第一卷积池化单元和N个第二卷积池化单元，再提取目标图像新的区域卷积特征信息，不断地迭代更新原始掩膜。
下面说明第二单位模型中包括多个卷积池化层时如何更新原始掩膜：
若第二单位模型中包含多个卷积池化层，那么对应的第一单位模型中也包含多个池化层，第二单位模型中包含卷积池化层的数量＝第一单位模型中包含池化层的数量。
可以将第二单位模型中的多个卷积池化层分为第一卷积池化层和第二卷积池化层，其中位于第二单位模型中顶部的卷积池化层是第一卷积池化层，其余的卷积池化层都是第二卷积池化层；那么对应地，原始掩膜就包括与第一卷积池化层对应的第一原始掩膜以及与每个第二卷积池化层分别对应的第二原始掩膜，可以理解为第一原始掩膜是由第一单位模型中的第一池化层生成的，第二原始掩膜是由第一单元模型中的第二池化层生成的。
将目标图像和第一原始掩膜输入第一卷积池化层，对目标图像进行卷积和池化，生成第一卷积特征信息。
此时检测第二卷积池化层的数量是否只有1个；
若此时第二卷积池化层的数量只有1个(对应的此时第二原始掩膜的数量也只有1个)，将第一卷积特征信息和第二原始掩膜输入第二卷积池化层，对第一卷积特征信息进行卷积和池化，得到目标图像的区域卷积特征信息。
若此时第二卷积池化层的数量不止1个(对应的此时第二原始掩膜的数量也不止1个)，将与当前第一卷积池化层相邻的第二卷积池化层作为新的第一卷积池化层，将第一卷积特征信息作为目标图像，将与第一原始掩膜相邻的第二原始掩膜作为新的第一原始掩膜。将新的目标图像和新的第一原始掩膜输入新的第一卷积池化层，对新的目标图像进行卷积和池化，生成新的第一卷积特征信息。此时再检测第二卷积池化层的数量是否只有1个，若是，将新的第一卷积特征信息和剩余的1个第二原始掩膜输入第二卷积池化层，对新的第一卷积特征信息进行卷积和池化，得到目标图像的区域卷积特征信息；若否，再将与当前第一卷积池化层相邻的第二卷积池化层作为新的第一卷积池化层，将新生成的第一卷积特征信息作为新的目标图像，将与当前第一原始掩膜相邻的第二原始掩膜作为新的第一原始掩膜，不断的循环，直至所有的第二卷积池化层都参与了运算，最后一个第二卷积池化层输出目标图像的区域卷积特征信息。
相当于，根据多个第二单位模型的多个卷积池化层，生成了多个第一卷积特征信息和一个区域卷积特征信息，第一卷积特征信息的数量+区域卷积特征信息的数量＝第二单位模型所包含的卷积池化层的数量。
将最后输出的区域卷积特征信息进行反卷积，得到第三卷积特征信息，其中第三卷积特征信息的特征图尺寸与近邻第一卷积特征信息的特征图尺寸相同，但第三卷积特征信息的通道数与近邻第一卷积特征信息的通道数可以不同，近邻第一卷积特征信息对应的卷积池化层与当前的区域卷积特征信息对应的卷积池化层相邻。
检测当前除近邻第一卷积特征信息外，是否还存在第一卷积特征信息未进行叠加运算(此处的叠加运算是指第一卷积特征信息和第三卷积特征信息之间的叠加)，若不存在，将第三卷积特征信息和近邻第一卷积特征信息在通道数的维度上进行叠加，再将叠加后的卷积特征信息进行卷积运算，得到目标卷积特征信息。例如，若第三卷积特征信息的尺寸是：50×50×100，近邻第一卷积特征信息的尺寸是：50×50×140，叠加后的卷积特征信息的尺寸是：50×50×240。
若当前除近邻第一卷积特征信息外，还存在第一卷积特征信息未进行叠加运算，将第三卷积特征信息和近邻第一卷积特征信息在通道数的维度上进行叠加和卷积运算，得到参考卷积特征信息。将得到的参考卷积特征信息作为新的区域卷积特征信息，再将与新的区域卷积特征信息对应的卷积池化层相邻的卷积池化层对应的第一卷积特征信息作为新的近邻第一卷积特征信息。
再将新的区域卷积特征信息进行反卷积，得到新的第三卷积特征信息，当然新的第三卷积特征信息的特征图尺寸同样等于新的近邻第一卷积特征信息的特征图尺寸。
再检测当前除近邻第一卷积特征信息外，是否还存在第一卷积特征信息未进行叠加运算，若不存在，将新的第三卷积特征信息和新的近邻第一卷积特征信息在通道数的维度上进行叠加和卷积运算，得到目标卷积特征信息。
若当前除近邻第一卷积特征信息外，还存在第一卷积特征信息未进行叠加运算，将新的第三卷积特征信息和新的近邻第一卷积特征信息在通道数的维度上进行叠加和卷积运算，得到新的参考卷积特征信息。
不断的迭代，直至所有的第一卷积特征信息都参与了叠加运算，最后输出目标卷积特征信息。
得到了目标卷积特征信息后，再将目标卷积特征信息进行反卷积运算，得到反卷积特征信息，反卷积特征信息的特征图尺寸和目标图像的尺寸相同。
再将反卷积特征信息进行全连接，全连接可以是计算多个通道的特征图在同一个位置的平均值，输出待更新掩膜，再将该待更新掩膜再转换为二值掩膜(二值掩膜中的取值要么是第一数值，要么是第二数值)，再将该二值掩膜作为新的原始掩膜。
至此，就完成了对原始掩膜的一次迭代更新。
再执行上述过程，即将新的原始掩膜输入第一单位模型，基于多个池化层进行下采样，得到新的下采样掩膜；将目标图像输入第二单位模型，根据新的下采样掩膜和第二单位模型中的多个卷积池化层，再提取目标图像新的区域卷积特征信息，不断地迭代更新原始掩膜。
同样地，第二单位模型中的每个卷积池化层都包括一个第一卷积池化单元和N个第二卷积池化单元，N是大于0的整数，其中基于每个卷积池化层生成第一卷积特征信息，以及基于最后一个位于底层的卷积池化层输出目标图像的区域特征信息的具体过程可以参见上述描述，即，当第二单位模型中只包括一个卷积池化层时，卷积池化层如何确定区域卷积特征信息，当然上述是一个卷积池化层就直接输出区域卷积特征信息，若第二单位模型包括多个卷积池化层，中间会输出多个第一卷积特征信息，最后一个卷积池化层才会输出区域卷积特征信息，虽然输入数据不同，卷积池化层所涉及的计算过程都是相同的。
步骤S104，当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
具体的，从上述描述可知，每一次迭代更新原始掩膜的实质就是生成一个新的二值掩膜， 将新的二值掩膜作为新的原始掩膜，再输入第二单位模型。但在这个迭代更新过程中，第一单位模型、第二单位模型以及目标图像都是不变的，发生变化的只是原始掩膜。
终端设备获取更新次数阈值，若更新原始掩膜的次数达到更新次数阈值，则说明最后一次更新的原始掩膜满足误差收敛条件，那么终端设备可以将最后一次更新的原始掩膜作为目标掩膜(如上述图2a-图2b对应实施例中的目标掩膜20g)。
或者，终端设备计算更新前的原始掩膜和更新后的原始掩膜之间的误差，检测该误差是否小于预设的误差阈值，若是，则说明更新后的原始掩膜满足误差收敛条件，此时终端设备可以将更新后的原始掩膜作为目标掩膜；若否，说明更新后的原始掩膜还未满足误差收敛条件，说明需要继续迭代更新原始掩膜，直至更新后的原始掩膜满足误差收敛条件。
下面说明如何计算更新前的原始掩膜和更新后的原始掩膜之间的误差：
记更新前的原始掩膜为mask1，更新后的原始掩膜为mask2，目标图像为I s，根据下述公式(4)可以确定第一误差l1：
Figure PCTCN2020115209-appb-000003
其中，|·|表示矩阵中的元素相加。
根据更新前的原始掩膜为mask1，更新后的原始掩膜为mask2，以及下述公式(5)可以确定第二误差l2：
Figure PCTCN2020115209-appb-000004
其中，|mask1∩mask2|表示将更新前的原始掩膜mask1和更新后的原始掩膜mask2进行点乘，然后再将矩阵中的元素相加。
将第一误差l1和第二误差l2相加，得到更新前的原始掩膜和更新后的原始掩膜之间的误差l：
l＝l1+l2         (6)
至此，终端设备确定了目标掩膜，从前述可以知道，目标掩膜中的单位目标掩膜的取值要么是小于掩膜阈值的第一数值，要么是大于掩膜阈值的第二数值。目标掩膜的尺寸和目标图像的尺寸相同，且目标掩膜的每个单位掩膜和目标图像的每个像素点具有一一对应关系。
目标掩膜的意义是：若单位目标掩膜的取值为第一数值，说明该单位目标掩膜对应的像素点的属性是背景属性；若单位目标掩膜的取值为第二数值，说明该单位目标掩膜对应的像素点的属性是与目标对象对应的目标对象属性。
举例来说，若目标掩膜为：
Figure PCTCN2020115209-appb-000005
上述目标掩膜包含4个单位目标掩膜，若目标对象是病灶对象，说明目标图像的第一个像素点(左上角像素点)的属性是背景属性，第二像素点和第三个像素点的属性是病灶对象属性，第四个像素点(右下角像素点)的属性是背景属性，也 即是第一个像素点和第四个像素点是背景像素点，第二个像素点和第三个像素点是病灶对象像素点。
换句话说，通过目标掩膜可以确定目标对象的每一个像素点所属的类别。
在目标掩膜中，终端设备确定大于掩膜阈值的单位目标掩膜的位置信息，在目标图像中，将该位置信息对应的图像区域作为目标对象在目标图像中的区域，称为目标图像区域(如上述图2a-图2b对应实施例中的小狗所在的图像区域)。
若目标图像是生物组织图像，目标对象是病灶对象，那么包含病灶对象的目标图像区域即是病灶图像区域。
终端设备可以在目标图像中，将目标图像区域标识出来，例如，用虚线将目标图像区域的边界标识出来，或者高亮显示目标图像区域等。
需要说明的是，目标对象的数量可以是1个也可以是多个，若目标对象是1个(属于两分类问题)，那么对应的原始掩膜以及目标掩膜的数量就只有1个，目标掩膜中的每个单位目标掩膜可以表示对应像素点的属性要么是背景属性要么是目标对象属性。
若目标对象是多个(属于多分类问题)，那么对应的原始掩膜的数量以及输出的目标掩膜的数量也应该是多个，每一个目标掩膜中的每个单位目标掩膜可以表示对应像素点属于第k个目标对象属性的概率，当然若某个像素点不属于任何一个目标对象属性，那么该像素点属于背景属性。
请参见图7，是本申请实施例提供的一种图像处理方法的模型架构图，以第一单位模型包括4个池化层，第二单位模型包括4个卷积池化层为例进行说明。
目标图像的尺寸为192*168*128，说明目标图像是三维图像，对应地，原始掩膜是尺寸为192*168*128的全0三维图像。将原始掩膜输入池化层，对原始掩膜进行最大池化，得到尺寸为96*84*64的下采样掩膜1、尺寸为48*42*32的下采样掩膜2、尺寸为24*21*16的下采样掩膜3以及尺寸为12*11*8的下采样掩膜4。
将目标图像输入卷积池化层(每个卷积池化层包括1个卷积池化单元和2个依赖卷积单元，即可对应前述中的1个第一卷积池化单元和2个第二卷积池化单元)，提取目标图像的特征信息，分别得到尺寸为96*84*64，64的第一卷积特征信息1、尺寸为48*42*32，128的第一卷积特征信息2、尺寸为24*21*16，256的第一卷积特征信息3以及尺寸为12*11*8，512的区域卷积特征信息4，其中上述4个卷积特征信息中，逗号前面的a*b*c表示特征图尺寸，逗号后面的数值表示通道数。
从图7可以看出，在每个卷积池化层中，卷积池化单元的输出数据的尺寸＝依赖卷积单元的输出数据的尺寸。
将尺寸为12*11*8，512的区域卷积特征信息4进行反卷积，得到尺寸为24*21*16，512的反卷积特征信息3，将反卷积特征信息3和尺寸为24*21*16，256的第一卷积特征信息3进行 叠加以及卷积，得到尺寸为24*21*16，512的叠加卷积特征信息3。
将尺寸为24*21*16，512的叠加卷积特征信息3进行反卷积，得到尺寸为48*42*32，512的反卷积特征信息2，将反卷积特征信息2和尺寸为48*42*32，128的第一卷积特征信息2进行叠加以及卷积，得到尺寸为48*42*32，256的叠加卷积特征信息2。
将尺寸为48*42*32，256的叠加卷积特征信息2进行反卷积，得到尺寸为96*84*64，256的反卷积特征信息1，将反卷积特征信息1和尺寸为96*84*64，64的第一卷积特征信息1进行叠加以及卷积，得到尺寸为96*84*64，128的目标卷积特征信息。
将尺寸为96*84*64，128的目标卷积特征信息反卷积，得到尺寸为192*168*128，128的反卷积特征信息。
计算反卷积特征信息在每个通道上的平均值(即是全连接)，得到尺寸为192*168*128的待更新掩膜。
再将上述尺寸为192*168*128的待更新掩膜作为新的原始掩膜，不断更新。当更新停止时，可以根据最新的原始掩膜确定目标对象在目标图像中的区域。
请参见图8，是本申请实施例提供的一种图像处理方法的流程示意图，可由图11所示的计算机设备执行，所述图像处理方法包括如下步骤：
步骤S201，获取包含所述目标对象的样本图像，获取样本原始掩膜和样本图像分割模型；所述样本图像分割模型包括所述第一单位模型和样本第二单位模型。
下面对如何训练图像分割模型进行说明，从前述中可知图像分割模型包括第一单位模型和第二单位模型，且第一单位模型中只包含池化层，因此第一单位模型不存在模型变量参数需要训练，即第一单位模型不需要训练，只需要训练第二单位模型即可。
终端设备获取包含目标对象的样本图像，获取样本原始掩膜和样本图像分割模型，其中样本原始掩膜中的每一个单位样本原始掩膜的取值可以都为第一数值，即是数值0。
样本图像分割模型包括第一单位模型和样本第二单位模型。
步骤S202，基于所述第一单位模型中的池化层，对所述样本原始掩膜进行下采样，得到样本下采样掩膜。
具体的，和前述第一单位模型的使用一样，基于第一单位模型中的池化层，对样本原始掩膜下采样，得到样本下采样掩膜。
步骤S203，基于所述样本第二单位模型和所述样本下采样掩膜，生成预测掩膜，根据所述目标对象在所述样本图像中的图像区域，确定样本掩膜。
具体的，基于前向传播，将样本图像和样本下采样掩膜，输入样本第二单位模型，生成预测掩膜mask pre。其中，前向传播生成预测掩膜mask pre的具体过程和前述中生成目标掩膜的过程相同。
确定目标对象在样本图像中的真实图像区域，根据该图像区域生成样本掩膜mask t，即样 本掩膜是由真实区域所确定的真实掩膜。
步骤S204，根据所述样本图像、所述预测掩膜和所述样本掩膜训练所述样本第二单位模型，得到所述图像分割模型。
具体的，将预测掩膜mask pre、样本图像I t以及样本掩膜mask t代入下述公式(7)，生成亮度损失intensityloss：
Figure PCTCN2020115209-appb-000006
将预测掩膜mask pre以及样本掩膜mask t代入下述公式(8)，生成分割损失diceloss：
Figure PCTCN2020115209-appb-000007
将上述亮度损失intensityloss以及分割损失diceloss组合为目标损失targetloss：
targetloss＝diceloss+intensityloss        (9)
基于梯度下降规则和目标损失，反向传播调整样本第二单位模型中的模型变量参数，得到目标参数，目标参数是目标损失最小时样本第二单位模型中的模型变量参数的取值。将样本第二单位模型中的模型变量参数替换为目标参数，至此就完成了对样本第二单位模型的模型变量参数的一次迭代更新。
再提取下一个包含目标对象的样本图像，再采用上述方式(当然此时的样本第二单位模型是已经更新过一次的样本第二单位模型)更新样本第二单位模型。
当更新次数达到预设的训练次数阈值，或者样本第二单位模型的模型变量参数更新前与更新后的差异量在预设范围内，说明样本第二单位模型训练完成，此时终端设备将训练完成的样本第二单位模型作为第二单位模型。
终端设备将上述训练好的第二单位模型和第一单位模型组合为图像分割模型。
步骤S205，获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型，基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜。
步骤S206，基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜。
步骤S207，当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
其中，步骤S205-步骤S207的具体实现过程可以参见上述图3对应实施例中的步骤S101-步骤S104。
请参见图9a，是本申请实施例提供的一种图像处理的功能模块图，如图9a所示，当目标图像是医学图像，目标对象是病灶对象时，前端A可以接收用户输入的待分割的医学图像，基于 图像预处理规则(例如，图像归一化、图像平移和/或图像旋转等)对医学图像进行预处理，得到预处理图像。将预处理图像传入后台，后台存储了已经训练好的图像分割模型。基于图像分割模型确定病灶对象在医学图像中的区域，后台可以将识别到的区域从医学图像中分割出来，得到病灶图像，后台将病灶图像发送至前端B(前端B与前端A可以是同一个前端)，前端B可以展示病灶图像或者对病灶图像进一步分析。
请参见图9b，是本申请实施例提供的一种电子医疗设备的结构示意图，电子医疗设备可以为上述图1-图9a对应实施例中的终端设备。电子医疗设备可以包括生物组织图像采集器和生物组织图像分析器，上述电子医疗设备可以采集医疗图像并分析医疗图像，具体过程包括如下步骤：
步骤S301，生物组织图像采集器获取包含病灶对象的生物组织图像。
具体的，生物组织图像采集器获取包含病灶对象的生物组织图像。若生物组织图像是MRI图像，生物组织图像采集器可以是MRI仪；若生物组织图像是CT图像，那么生物组织图像采集器可以是CT机；若生物组织图像是乳腺钼靶图像，那么生物组织图像采集器可以是钼靶机。
步骤S302，生物组织图像分析器获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；生物组织图像分析器基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜。
具体的，生物组织图像分析器获取训练好的图像分割模型，以及获取与生物组织图像具有相同尺寸的原始掩膜。
生物组织图像分析器将原始掩膜输入第一单位模型，基于第一单位模型中的池化层，对原始掩膜进行池化运算，得到下采样掩膜，其中此处的池化运算可以是最大池化运算也可以是平均池化运算。
生物组织图像分析器基于第一单位模型生成下采样掩膜的具体过程可以参见上述图3对应实施例中的步骤S102。
步骤S303，生物组织图像分析器基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述生物组织图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜。
具体的，生物组织图像分析器将下采样掩膜和生物组织图像输入第二单位模型，基于第二单位模型中的卷积池化层提取生物组织图像的区域卷积特征信息，根据区域卷积特征信息生成待更新掩膜，待更新掩膜二值化，得到二值掩膜，其中待更新掩膜的尺寸与二值掩膜的尺寸，以及原始掩膜的尺寸相同。生物组织图像分析器将生成的二值掩膜作为新的原始掩膜再次输入第一单位模型，不断迭代更新原始掩膜。
生物组织图像分析器基于第二单位模型更新原始掩膜的具体过程可以参见上述图3对应实施例中的步骤S103。
步骤S304，当更新后的原始掩膜满足误差收敛条件时，所述生物组织图像分析器根据更新 后的原始掩膜确定所述病灶对象在所述生物组织图像中的病灶图像区域。
具体的，若更新原始掩膜的次数达到更新次数阈值，或者，更新前的原始掩膜和更新后的原始掩膜之间的误差小于预设的误差阈值，则生物组织图像分析器将更新后的原始掩膜作为目标掩膜。在目标掩膜中，生物组织图像分析器确定大于掩膜阈值的单位目标掩膜的位置信息，在生物组织图像中，生物组织图像分析器将该位置信息对应的图像区域作为病灶对象在生物组织图像中的区域，称为病灶图像区域。
后续，生物组织图像显示器可以将病灶图像区域在生物组织图像中标识出来，并展示标识了病灶图像区域的生物组织图像。
进一步的，请参见图10，是本申请实施例提供的一种图像处理装置的结构示意图。如图10所示，图像处理装置1可以应用于上述图1-图9对应实施例中的终端设备，图像处理装置1可以包括：图像获取模块11、模型获取模块12、池化模块13、卷积模块14、更新模块15以及掩膜确定模块16。
图像获取模块11，用于获取包含目标对象的目标图像；
模型获取模块12，用于获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
池化模块13，用于基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
卷积模块14，用于基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息；
更新模块15，用于根据所述区域卷积特征信息更新所述原始掩膜；
掩膜确定模块16，用于当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域；
图像获取模块11，具体用于获取包含所述目标对象的原始图像，基于图像预处理规则对所述原始图像进行图像预处理，得到所述目标图像；所述图像预处理规则包括图像归一化、图像平移、图像旋转、图像对称以及图像缩放；
掩膜确定模块16，具体用于将更新后的原始掩膜确定为目标掩膜，在所述目标掩膜中，确定大于掩膜阈值的单位目标掩膜的位置信息，在所述目标图像中，将与所述位置信息对应的图像区域作为所述目标图像区域；所述目标掩膜包括多个单位目标掩膜。
其中，图像获取模块11、模型获取模块12、池化模块13、卷积模块14、更新模块15以及掩膜确定模块16的具体功能实现方式可以参见上述图3对应实施例中的步骤S101-步骤S104，这里不再进行赘述。
请参见图10，第二单位模型中的卷积池化层包括第一卷积池化层和第二卷积池化层；所述下采样掩膜包括与所述第一卷积池化层对应的第一原始掩膜以及与所述第二卷积池化层对应的第二原始掩膜；
卷积模块14可以包括：第一卷积单元141以及第二卷积单元142。
第一卷积单元141，用于基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息；
第二卷积单元142，用于基于所述第二卷积池化层和所述第二原始掩膜，对所述第一卷积特征信息进行卷积和池化，得到所述目标图像的所述区域卷积特征信息；
其中，第一卷积单元141以及第二卷积单元142的具体过程可以参见上述图3对应实施例中的步骤S103。
请参见图10，所述第一卷积池化层包括第一卷积池化单元以及第二卷积池化单元；
第一卷积单元141，具体用于基于所述第一卷积池化单元对应的卷积函数和池化函数，对所述目标图像进行卷积和池化，得到输入卷积特征信息，基于所述第二卷积池化单元对应的卷积函数，对所述输入卷积特征信息进行编码，生成第二卷积特征信息，根据所述第一原始掩膜，对所述第二卷积特征信息的多个通道的特征图分别进行池化，确定池化向量，基于所述第二卷积池化单元对应的激活函数将所述池化向量转换为目标向量，根据所述目标向量和所述第一原始掩膜，生成所述第一卷积特征信息；所述输入卷积特征信息的尺寸、所述第一卷积特征信息的尺寸以及所述第二卷积特征信息的尺寸均相同。
其中，第一卷积单元141的具体过程可以参见上述图3对应实施例中的步骤S103。
请参见图10，更新模块15可以包括：反卷积单元151、叠加单元152以及确定单元153。
反卷积单元151，用于将所述区域卷积特征信息进行反卷积，生成第三卷积特征信息；
叠加单元152，用于将所述第三卷积特征信息和所述第一卷积特征信息叠加为目标卷积特征信息，将所述目标卷积特征信息进行反卷积和全连接，得到待更新掩膜；
确定单元153，用于将所述待更新掩膜确定为所述原始掩膜；所述待更新掩膜的尺寸与所述目标图像的尺寸相同；
所述待更新掩膜包括多个单位待更新掩膜；
所述确定单元153，具体用于在所述待更新掩膜中，将小于或等于掩膜阈值的单位待更新掩膜的取值调整为第一数值，在所述待更新掩膜中，将大于所述掩膜阈值的单位待更新掩膜的取值调整为第二数值，得到所述原始掩膜。
其中，反卷积单元151、叠加单元152以及确定单元153的具体过程可以参见上述图3对应实施例中的步骤S103，此处不再赘述。
图像处理装置1可以包括：图像获取模块11、模型获取模块12、池化模块13、卷积模块14、更新模块15以及掩膜确定模块16；还可以包括：收敛确定模块17。
收敛确定模块17，用于若更新后的原始掩膜与更新前的原始掩膜之间的误差小于误差阈值，则确定更新后的原始掩膜满足所述误差收敛条件，或，
所述收敛确定模块17，还用于若更新次数达到更新次数阈值，则确定更新后的原始掩膜满足所述误差收敛条件。
其中，收敛确定模块17的具体过程可以参见上述图3对应实施例中的步骤S104，此处不再赘述。
请进一步参见图10，图像处理装置1可以包括：图像获取模块11、模型获取模块12、池化模块13、卷积模块14、更新模块15以及掩膜确定模块16；还可以包括：样本获取模块18以及训练模块19。
样本获取模块18，用于获取包含所述目标对象的样本图像，获取样本原始掩膜和样本图像分割模型；所述样本图像分割模型包括所述第一单位模型和样本第二单位模型；
所述样本获取模块18，还用于基于所述第一单位模型中的池化层，对所述样本原始掩膜进行下采样，得到样本下采样掩膜；
所述样本获取模块18，还用于基于所述样本第二单位模型和所述样本下采样掩膜，生成预测掩膜，根据所述目标对象在所述样本图像中的图像区域，确定样本掩膜；
训练模块19，用于根据所述样本图像、所述预测掩膜和所述样本掩膜训练所述样本第二单位模型，得到所述图像分割模型；
训练模块19，具体用于根据所述样本图像、所述预测掩膜和所述样本掩膜，生成亮度损失，根据所述预测掩膜和所述样本掩膜，生成分割损失，将所述亮度损失和所述分割损失组合为目标损失，基于梯度下降规则和所述目标损失，确定所述样本第二单位模型中的模型变量参数的参数值，根据所述参数值更新所述样本第二单位模型中的模型变量参数，当训练次数达到训练次数阈值时，将模型变量参数更新后的样本第二单位模型作为所述第二单位模型，将所述第一单位模型和所述第二单位模型组合为所述图像分割模型。
其中，样本获取模块18以及训练模块19的具体过程可以参见上述图8对应实施例中的步骤S201-步骤S204，此处不再赘述
目标对象是病灶对象；所述目标图像是生物组织图像；所述目标图像区域是病灶图像区域。
本申请通过获取包含目标对象的目标图像，并获取原始掩膜以及图像分割模型，基于图像分割模型中的第一单位模型，对原始掩膜下采样，得到下采样掩膜；基于图像分割模型中的第二单位模型以及下采样掩膜，提取目标图像的区域特征信息，根据区域特征信息更新原始掩膜，不断地的迭代更新原始掩膜，直至更新后的原始掩膜满足收敛条件时，根据更新后的原始掩膜可以确定目标对象在目标图像中的区域。上述可知，通过自动化地方式自动分割出目标对象在图像中的区域，相比人工分割，可以降低图像分割的耗时，提高图像分割的效率。
进一步地，请参见图11，是本申请实施例提供的一种计算机设备的结构示意图。上述图1-图9对应实施例中的终端设备可以为计算机设备1000，如图11所示，所述计算机设备1000可以包括：用户接口1002、处理器1004、编码器1006以及存储器1008。信号接收器1016用于经由蜂窝接口1010、WIFI接口1012、...、或NFC接口1014接收或者发送数据。编码器1006将接收到的数据编码为计算机处理的数据格式。存储器1008中存储有计算机程序，处理器1004被设置为通过计算机程序执行上述任一项方法实施例中的步骤。存储器1008可包括易失性存储 器(例如，动态随机存取存储器DRAM)，还可以包括非易失性存储器(例如，一次性可编程只读存储器OTPROM)。在一些实例中，存储器1008可进一步包括相对于处理器1004远程设置的存储器，这些远程存储器可以通过网络连接至计算机设备1000。用户接口1002可以包括：键盘1018和显示器1020。
在图11所示的计算机设备1000中，处理器1004可以用于调用存储器1008中存储的计算机程序，以实现：
获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜；
当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
在一个实施例中，所述第二单位模型中的卷积池化层包括第一卷积池化层和第二卷积池化层；所述下采样掩膜包括与所述第一卷积池化层对应的第一原始掩膜以及与所述第二卷积池化层对应的第二原始掩膜；
处理器1004在执行基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息时，具体执行以下步骤：
基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息；
基于所述第二卷积池化层和所述第二原始掩膜，对所述第一卷积特征信息进行卷积和池化，得到所述目标图像的所述区域卷积特征信息。
在一个实施例中，第一卷积池化层包括第一卷积池化单元以及第二卷积池化单元；
处理器1004在执行基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息时，具体执行以下步骤：
基于所述第一卷积池化单元对应的卷积函数和池化函数，对所述目标图像进行卷积和池化，得到输入卷积特征信息；
基于所述第二卷积池化单元对应的卷积函数，对所述输入卷积特征信息进行编码，生成第二卷积特征信息；
根据所述第一原始掩膜，对所述第二卷积特征信息的的多个通道的特征图分别进行池化，确定池化向量；
基于所述第二卷积池化单元对应的激活函数将所述池化向量转换为目标向量；
根据所述目标向量和所述第一原始掩膜，生成所述第一卷积特征信息；所述输入卷积特征信息的尺寸、所述第一卷积特征信息的尺寸以及所述第二卷积特征信息的尺寸均相同。
在一个实施例中，处理器1004在执行根据所述区域卷积特征信息更新所述原始掩膜时，具体执行以下步骤：
将所述区域卷积特征信息进行反卷积，生成第三卷积特征信息；
将所述第三卷积特征信息和所述第一卷积特征信息叠加为目标卷积特征信息；
将所述目标卷积特征信息进行反卷积和全连接，得到待更新掩膜，将所述待更新掩膜确定为所述原始掩膜；所述待更新掩膜的尺寸与所述目标图像的尺寸相同。
在一个实施例中，待更新掩膜包括多个单位待更新掩膜；
处理器1004在执行将所述待更新掩膜确定为所述原始掩膜时，具体执行以下步骤：
在所述待更新掩膜中，将小于或等于掩膜阈值的单位待更新掩膜的取值调整为第一数值；
在所述待更新掩膜中，将大于所述掩膜阈值的单位待更新掩膜的取值调整为第二数值，得到所述原始掩膜。
在一个实施例中，处理器1004在执行根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域时，具体执行以下步骤：
将更新后的原始掩膜确定为目标掩膜；所述目标掩膜包括多个单位目标掩膜；
在所述目标掩膜中，确定大于掩膜阈值的单位目标掩膜的位置信息；
在所述目标图像中，将与所述位置信息对应的图像区域作为所述目标图像区域。
在一个实施例中，处理器1004在执行获取包含目标对象的目标图像时，具体执行以下步骤：
获取包含所述目标对象的原始图像；
基于图像预处理规则对所述原始图像进行图像预处理，得到所述目标图像；所述图像预处理规则包括图像归一化、图像平移、图像旋转、图像对称以及图像缩放。
在一个实施例中，处理器1004还执行以下步骤：
若更新后的原始掩膜与更新前的原始掩膜之间的误差小于误差阈值，则确定更新后的原始掩膜满足所述误差收敛条件，或，
若更新次数达到更新次数阈值，则确定更新后的原始掩膜满足所述误差收敛条件。
在一个实施例中，处理器1004还执行以下步骤：
获取包含所述目标对象的样本图像，获取样本原始掩膜和样本图像分割模型；所述样本图像分割模型包括所述第一单位模型和样本第二单位模型；
基于所述第一单位模型中的池化层，对所述样本原始掩膜进行下采样，得到样本下采样掩膜；
基于所述样本第二单位模型和所述样本下采样掩膜，生成预测掩膜；
根据所述目标对象在所述样本图像中的图像区域，确定样本掩膜；
根据所述样本图像、所述预测掩膜和所述样本掩膜训练所述样本第二单位模型，得到所述图像分割模型。
在一个实施例中，处理器1004在执行根据所述样本图像、所述预测掩膜和所述样本掩膜训 练所述样本第二单位模型，得到所述图像分割模型时，具体执行以下步骤：
根据所述样本图像、所述预测掩膜和所述样本掩膜，生成亮度损失；
根据所述预测掩膜和所述样本掩膜，生成分割损失；
将所述亮度损失和所述分割损失组合为目标损失；
基于梯度下降规则和所述目标损失，确定所述样本第二单位模型中的模型变量参数的参数值，根据所述参数值更新所述样本第二单位模型中的模型变量参数；
当训练次数达到训练次数阈值时，将模型变量参数更新后的样本第二单位模型作为所述第二单位模型，将所述第一单位模型和所述第二单位模型组合为所述图像分割模型。
在一个实施例中，所述目标对象是病灶对象；所述目标图像是生物组织图像；所述目标图像区域是病灶图像区域。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图1到图9所对应实施例中对所述图像处理方法的描述，也可执行前文图10所对应实施例中对所述图像处理装置1的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
根据本申请的一个方面，提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。在图11所示的计算机设备1000中的处理器1004从计算机可读存储介质读取该计算机指令，处理器1004执行该计算机指令，使得该计算机设备1000执行上述各个方法实施例中所述的图像处理方法。
此外，这里需要指出的是：本申请实施例还提供了一种计算机存储介质，且所述计算机存储介质中存储有前文提及的图像处理装置1所执行的计算机程序，且所述计算机程序包括程序指令，当所述处理器执行所述程序指令时，能够执行前文图1到图9所对应实施例中对所述图像处理方法的描述，因此，这里将不再进行赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。对于本申请所涉及的计算机存储介质实施例中未披露的技术细节，请参照本申请方法实施例的描述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (20)

  1. 一种图像处理方法，由计算机设备执行，包括：
    获取包含目标对象的目标图像，获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
    基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
    基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜；
    当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
  2. 根据权利要求1所述的方法，其中，所述第二单位模型中的卷积池化层包括第一卷积池化层和第二卷积池化层；所述下采样掩膜包括与所述第一卷积池化层对应的第一原始掩膜以及与所述第二卷积池化层对应的第二原始掩膜；
    所述基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息，包括：
    基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息；
    基于所述第二卷积池化层和所述第二原始掩膜，对所述第一卷积特征信息进行卷积和池化，得到所述目标图像的所述区域卷积特征信息。
  3. 根据权利要求2所述的方法，其中，所述第一卷积池化层包括第一卷积池化单元以及第二卷积池化单元；
    所述基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息，包括：
    基于所述第一卷积池化单元对应的卷积函数和池化函数，对所述目标图像进行卷积和池化，得到输入卷积特征信息；
    基于所述第二卷积池化单元对应的卷积函数，对所述输入卷积特征信息进行编码，生成第二卷积特征信息；
    根据所述第一原始掩膜，对所述第二卷积特征信息的多个通道的特征图分别进行池化，确定池化向量；
    基于所述第二卷积池化单元对应的激活函数，将所述池化向量转换为目标向量；
    根据所述目标向量和所述第一原始掩膜，生成所述第一卷积特征信息；所述输入卷积特征信息的尺寸、所述第一卷积特征信息的尺寸以及所述第二卷积特征信息的尺寸均相同。
  4. 根据权利要求2所述的方法，其中，所述根据所述区域卷积特征信息更新所述原始掩膜， 包括：
    将所述区域卷积特征信息进行反卷积，生成第三卷积特征信息；
    将所述第三卷积特征信息和所述第一卷积特征信息叠加为目标卷积特征信息；
    将所述目标卷积特征信息进行反卷积和全连接，得到待更新掩膜，将所述待更新掩膜确定为所述原始掩膜；所述待更新掩膜的尺寸与所述目标图像的尺寸相同。
  5. 根据权利要求4所述的方法，其中，所述待更新掩膜包括多个单位待更新掩膜；
    所述将所述待更新掩膜确定为所述原始掩膜，包括：
    在所述待更新掩膜中，将小于或等于掩膜阈值的单位待更新掩膜的取值调整为第一数值；
    在所述待更新掩膜中，将大于所述掩膜阈值的单位待更新掩膜的取值调整为第二数值，得到所述原始掩膜。
  6. 根据权利要求1所述的方法，其中，所述根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域，包括：
    将更新后的原始掩膜确定为目标掩膜；所述目标掩膜包括多个单位目标掩膜；
    在所述目标掩膜中，确定大于掩膜阈值的单位目标掩膜的位置信息；
    在所述目标图像中，将与所述位置信息对应的图像区域作为所述目标图像区域。
  7. 根据权利要求1所述的方法，其中，所述获取包含目标对象的目标图像，包括：
    获取包含所述目标对象的原始图像；
    基于图像预处理规则对所述原始图像进行图像预处理，得到所述目标图像；所述图像预处理规则包括图像归一化、图像平移、图像旋转、图像对称以及图像缩放。
  8. 根据权利要求1所述的方法，其中，还包括：
    若更新后的原始掩膜与更新前的原始掩膜之间的误差小于误差阈值，则确定更新后的原始掩膜满足所述误差收敛条件，或，
    若更新次数达到更新次数阈值，则确定更新后的原始掩膜满足所述误差收敛条件。
  9. 根据权利要求1所述的方法，其中，还包括：
    获取包含所述目标对象的样本图像，获取样本原始掩膜和样本图像分割模型；所述样本图像分割模型包括所述第一单位模型和样本第二单位模型；
    基于所述第一单位模型中的池化层，对所述样本原始掩膜进行下采样，得到样本下采样掩膜；
    基于所述样本第二单位模型和所述样本下采样掩膜，生成预测掩膜；
    根据所述目标对象在所述样本图像中的图像区域，确定样本掩膜；
    根据所述样本图像、所述预测掩膜和所述样本掩膜训练所述样本第二单位模型，得到所述图像分割模型。
  10. 根据权利要求9所述的方法，其中，所述根据所述样本图像、所述预测掩膜和所述样本掩膜训练所述样本第二单位模型，得到所述图像分割模型，包括：
    根据所述样本图像、所述预测掩膜和所述样本掩膜，生成亮度损失；
    根据所述预测掩膜和所述样本掩膜，生成分割损失；
    将所述亮度损失和所述分割损失组合为目标损失；
    基于梯度下降规则和所述目标损失，确定所述样本第二单位模型中的模型变量参数的参数值，根据所述参数值更新所述样本第二单位模型中的模型变量参数；
    当训练次数达到训练次数阈值时，将模型变量参数更新后的样本第二单位模型作为所述第二单位模型，将所述第一单位模型和所述第二单位模型组合为所述图像分割模型。
  11. 根据权利要求1所述的方法，其中，所述目标对象是病灶对象；所述目标图像是生物组织图像；所述目标图像区域是病灶图像区域。
  12. 一种图像处理装置，包括：
    图像获取模块，用于获取包含目标对象的目标图像；
    模型获取模块，用于获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
    池化模块，用于基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
    卷积模块，用于基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述目标图像的区域卷积特征信息；
    更新模块，用于根据所述区域卷积特征信息更新所述原始掩膜；
    掩膜确定模块，用于当更新后的原始掩膜满足误差收敛条件时，根据更新后的原始掩膜确定所述目标对象在所述目标图像中的目标图像区域。
  13. 根据权利要求12所述的装置，其中，所述卷积模块包括：
    第一卷积单元，用于基于所述第一卷积池化层和所述第一原始掩膜，对所述目标图像进行卷积和池化，得到第一卷积特征信息；
    第二卷积单元，用于基于所述第二卷积池化层和所述第二原始掩膜，对所述第一卷积特征信息进行卷积和池化，得到所述目标图像的所述区域卷积特征信息。
  14. 根据权利要求13所述的装置，其中，所述第一卷积单元进一步用于基于所述第一卷积池化单元对应的卷积函数和池化函数，对所述目标图像进行卷积和池化，得到输入卷积特征信息，基于所述第二卷积池化单元对应的卷积函数，对所述输入卷积特征信息进行编码，生成第二卷积特征信息，根据所述第一原始掩膜，对所述第二卷积特征信息的多个通道的特征图分别进行池化，确定池化向量，基于所述第二卷积池化单元对应的激活函数，将所述池化向量转换为目标向量，根据所述目标向量和所述第一原始掩膜，生成所述第一卷积特征信息；所述输入卷积特征信息的尺寸、所述第一卷积特征信息的尺寸以及所述第二卷积特征信息的尺寸均相同。
  15. 根据权利要求13所述的装置，其中，所述更新模块包括：
    反卷积单元，用于将所述区域卷积特征信息进行反卷积，生成第三卷积特征信息；
    叠加单元，用于将所述第三卷积特征信息和所述第一卷积特征信息叠加为目标卷积特征信息，将所述目标卷积特征信息进行反卷积和全连接，得到待更新掩膜；
    确定单元，用于将所述待更新掩膜确定为所述原始掩膜；所述待更新掩膜的尺寸与所述目标图像的尺寸相同。
  16. 根据权利要求15所述的装置，其中，所述待更新掩膜包括多个单位待更新掩膜；
    所述确定单元进一步用于在所述待更新掩膜中，将小于或等于掩膜阈值的单位待更新掩膜的取值调整为第一数值，在所述待更新掩膜中，将大于所述掩膜阈值的单位待更新掩膜的取值调整为第二数值，得到所述原始掩膜。
  17. 根据权利要求12所述的装置，其中，所述装置进一步包括：
    收敛确定模块，用于若更新后的原始掩膜与更新前的原始掩膜之间的误差小于误差阈值，则确定更新后的原始掩膜满足所述误差收敛条件，或，
    若更新次数达到更新次数阈值，则确定更新后的原始掩膜满足所述误差收敛条件。
  18. 一种电子医疗设备，包括生物组织图像采集器和生物组织图像分析器；
    所述生物组织图像采集器获取包含病灶对象的生物组织图像；
    所述生物组织图像分析器获取原始掩膜以及图像分割模型；所述图像分割模型包括第一单位模型和第二单位模型；
    所述生物组织图像分析器基于所述第一单位模型中的池化层，对所述原始掩膜进行下采样，得到下采样掩膜；
    所述生物组织图像分析器基于所述第二单位模型中的卷积池化层以及所述下采样掩膜，提取所述生物组织图像的区域卷积特征信息，根据所述区域卷积特征信息更新所述原始掩膜；
    当更新后的原始掩膜满足误差收敛条件时，所述生物组织图像分析器根据更新后的原始掩膜确定所述病灶对象在所述生物组织图像中的病灶图像区域。
  19. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如权利要求1-11中任一项所述方法的步骤。
  20. 一种计算机存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时，执行如权利要求1-11任一项所述的方法。
PCT/CN2020/115209 2019-09-25 2020-09-15 一种图像处理方法、装置、计算机设备以及存储介质 WO2021057536A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/499,993 US20220028087A1 (en) 2019-09-25 2021-10-13 Image processing method and apparatus, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910912096.1 2019-09-25
CN201910912096.1A CN110675385B (zh) 2019-09-25 2019-09-25 一种图像处理方法、装置、计算机设备以及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/499,993 Continuation US20220028087A1 (en) 2019-09-25 2021-10-13 Image processing method and apparatus, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2021057536A1 true WO2021057536A1 (zh) 2021-04-01

Family

ID=69079011

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/115209 WO2021057536A1 (zh) 2019-09-25 2020-09-15 一种图像处理方法、装置、计算机设备以及存储介质

Country Status (3)

Country Link
US (1) US20220028087A1 (zh)
CN (1) CN110675385B (zh)
WO (1) WO2021057536A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110675385B (zh) * 2019-09-25 2024-05-14 腾讯科技（深圳）有限公司 一种图像处理方法、装置、计算机设备以及存储介质
US11507831B2 (en) 2020-02-24 2022-11-22 Stmicroelectronics International N.V. Pooling unit for deep learning acceleration
CN111429380B (zh) * 2020-04-08 2023-11-03 京东科技信息技术有限公司 一种图像矫正方法、装置、电子设备及存储介质
KR102179584B1 (ko) * 2020-04-16 2020-11-18 주식회사 딥노이드 의료영상 판독을 위한 인공지능 기반의 클라우드 플랫폼 시스템
CN111626187B (zh) * 2020-05-25 2023-08-08 京东科技信息技术有限公司 一种身份标注方法、装置、电子设备及存储介质
CN113608805B (zh) * 2021-07-08 2024-04-12 阿里巴巴创新公司 掩膜预测方法、图像处理方法、显示方法及设备
CN115272206B (zh) * 2022-07-18 2023-07-04 深圳市医未医疗科技有限公司 医学图像处理方法、装置、计算机设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160328630A1 (en) * 2015-05-08 2016-11-10 Samsung Electronics Co., Ltd. Object recognition apparatus and method
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
CN109389557A (zh) * 2018-10-20 2019-02-26 南京大学 一种基于图像先验的细胞图像超分辨方法及装置
CN109584246A (zh) * 2018-11-16 2019-04-05 成都信息工程大学 基于多尺度特征金字塔的dcm心肌诊疗放射影像分割方法
CN109978037A (zh) * 2019-03-18 2019-07-05 腾讯科技（深圳）有限公司 图像处理方法、模型训练方法、装置、和存储介质
CN110675385A (zh) * 2019-09-25 2020-01-10 腾讯科技（深圳）有限公司 一种图像处理方法、装置、计算机设备以及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108537292B (zh) * 2018-04-10 2020-07-31 上海白泽网络科技有限公司 语义分割网络训练方法、图像语义分割方法及装置
CN109493350B (zh) * 2018-11-09 2020-09-22 重庆中科云从科技有限公司 人像分割方法及装置
CN109886273B (zh) * 2019-02-26 2022-12-16 四川大学华西医院 一种cmr图像分割分类系统
CN109977798B (zh) * 2019-03-06 2021-06-04 中山大学 用于行人重识别的掩膜池化模型训练和行人重识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160328630A1 (en) * 2015-05-08 2016-11-10 Samsung Electronics Co., Ltd. Object recognition apparatus and method
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
CN109389557A (zh) * 2018-10-20 2019-02-26 南京大学 一种基于图像先验的细胞图像超分辨方法及装置
CN109584246A (zh) * 2018-11-16 2019-04-05 成都信息工程大学 基于多尺度特征金字塔的dcm心肌诊疗放射影像分割方法
CN109978037A (zh) * 2019-03-18 2019-07-05 腾讯科技（深圳）有限公司 图像处理方法、模型训练方法、装置、和存储介质
CN110675385A (zh) * 2019-09-25 2020-01-10 腾讯科技（深圳）有限公司 一种图像处理方法、装置、计算机设备以及存储介质

Also Published As

Publication number Publication date
CN110675385B (zh) 2024-05-14
CN110675385A (zh) 2020-01-10
US20220028087A1 (en) 2022-01-27

Similar Documents

Publication Publication Date Title
WO2021057536A1 (zh) 一种图像处理方法、装置、计算机设备以及存储介质
CN107516319B (zh) 一种高精度简易交互式抠图方法、存储设备及终端
US20210319560A1 (en) Image processing method and apparatus, and storage medium
KR101699919B1 (ko) 다중 노출 퓨전 기반에서 고스트 흐림을 제거한 ｈｄｒ 영상 생성 장치 및 방법
CN108304821B (zh) 图像识别方法及装置、图像获取方法及设备、计算机设备及非易失性计算机可读存储介质
US11526995B2 (en) Robust use of semantic segmentation for depth and disparity estimation
WO2022110638A1 (zh) 人像修复方法、装置、电子设备、存储介质和程序产品
CN115442515A (zh) 图像处理方法和设备
CN110493527B (zh) 主体对焦方法、装置、电子设备和存储介质
EP3664016B1 (en) Image detection method and apparatus, and terminal
CN113888437A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN114820584B (zh) 肺部病灶定位装置
WO2023020201A1 (zh) 图像的增强方法和电子设备
US20240046538A1 (en) Method for generating face shape adjustment image, model training method, apparatus and device
WO2023284401A1 (zh) 图像美颜处理方法、装置、存储介质与电子设备
US20220398698A1 (en) Image processing model generation method, processing method, storage medium, and terminal
WO2022261828A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN113283485A (zh) 目标检测方法及其模型的训练方法和相关装置及介质
US11032529B2 (en) Selectively applying color to an image
EP4093015A1 (en) Photographing method and apparatus, storage medium, and electronic device
CN113673474A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN113610884A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN110956063A (zh) 图像处理方法、装置、设备及存储介质
WO2024041108A1 (zh) 图像矫正模型训练及图像矫正方法、装置和计算机设备
CN115908120B (zh) 图像处理方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20869265

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20869265

Country of ref document: EP

Kind code of ref document: A1