CN110930296A - Image processing method, device, equipment and storage medium - Google Patents
Image processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN110930296A CN110930296A CN201911141216.9A CN201911141216A CN110930296A CN 110930296 A CN110930296 A CN 110930296A CN 201911141216 A CN201911141216 A CN 201911141216A CN 110930296 A CN110930296 A CN 110930296A
- Authority
- CN
- China
- Prior art keywords
- target
- image
- information
- network model
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 230000011218 segmentation Effects 0.000 claims abstract description 141
- 238000012545 processing Methods 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims description 27
- 210000004209 hair Anatomy 0.000 claims description 20
- 230000015654 memory Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 238000005260 corrosion Methods 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000003628 erosive effect Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000037308 hair color Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The embodiment of the application discloses an image processing method, an image processing device, image processing equipment and a storage medium, and belongs to the field of image processing. The method comprises the following steps: acquiring an image to be processed; the method comprises the steps that an image is used as input of a segmentation network model, and target segmentation information of the image is determined through the segmentation network model, wherein the target segmentation information comprises position information of a foreground region, position information of a background region and position information of an unknown region; the method comprises the steps that an image and target segmentation information of the image are used as input of a matting network model, target probability information of the image is determined through the matting network model, and the target probability information comprises the probability that each pixel point in the image belongs to a specified target; and processing the specified target in the image according to the target probability information of the image. According to the method and the device, the specified target can be finely divided and processed, the accuracy of target identification and the fineness of target processing are improved, and the image processing effect is further improved.
Description
Technical Field
The present disclosure relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a storage medium.
Background
In the field of image processing, it is often necessary to process a specified target in an image, for example, to perform a face beautifying process on a face in the image, or to perform a color conversion on hair of the face in the image, and the like. However, before processing the designated object in the image, the designated object needs to be accurately recognized from the image, so that the designated object can be processed.
Currently, an object segmentation algorithm is usually adopted to identify a specified object in an image. Specifically, an image to be processed and with a designated target is obtained, then a target segmentation algorithm is adopted to identify whether each pixel point in the image belongs to the designated target or belongs to a background outside the designated target, and an area surrounded by the pixel points belonging to the designated target is determined as a contour area of the designated target, so that the designated target is identified from the background.
However, when a target segmentation algorithm is used for target identification of an image, only an absolute identification result of whether each pixel belongs to or does not belong to a designated target can be given, but in an actual identification process, the shape of the designated target may be irregular, so that some pixels belonging to the designated target may be mistakenly identified as not belonging to the designated target, or some pixels not belonging to the designated target are mistakenly identified as belonging to the designated target, and further, an identified contour region is inaccurate, and an image processing effect is affected.
Disclosure of Invention
The embodiment of the application provides an image processing method, an image processing device, image processing equipment and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides an image processing method, where the method includes:
acquiring an image to be processed, wherein a specified target exists in the image;
the image is used as the input of a segmentation network model, the segmentation network model is used for determining the target segmentation information of the image, the target segmentation information comprises the position information of a foreground area identified as a specified target, the position information of a background area identified as a non-specified target and the position information of an unknown area which cannot be identified as a background or a foreground, and the segmentation network model is used for determining the target segmentation information of any image;
the image and the target segmentation information of the image are used as the input of a matting network model, the target probability information of the image is determined through the matting network model, the target probability information of the image comprises the probability that each pixel point in the image belongs to the specified target, and the matting network model is used for determining the target probability information of any image;
and processing the specified target in the image according to the target probability information of the image.
In another aspect, there is provided an image processing apparatus, the apparatus including:
the device comprises a first acquisition module, a second acquisition module and a processing module, wherein the first acquisition module is used for acquiring an image to be processed, and a specified target exists in the image;
a segmentation module, configured to use the image as an input of a segmentation network model, determine, by the segmentation network model, target segmentation information of the image, where the target segmentation information includes location information of a foreground region identified as a designated target, location information of a background region identified as a non-designated target, and location information of an unknown region that cannot be identified as a background or a foreground, and the segmentation network model is used to determine target segmentation information of any image;
the matting module is used for taking the image and the object segmentation information of the image as input of a matting network model, determining object probability information of the image through the matting network model, wherein the object probability information of the image comprises the probability that each pixel point in the image belongs to the specified object, and the matting network model is used for determining the object probability information of any image;
and the processing module is used for processing the specified target in the image according to the target probability information of the image.
In another aspect, an electronic device is provided, the electronic device comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the image processing method described above.
In another aspect, a computer-readable storage medium is provided, which stores at least one instruction for execution by a processor to implement the above-described image processing method.
In another aspect, a computer program product is provided, which stores at least one instruction for execution by a processor to implement the above-mentioned image processing method.
The technical scheme provided by the application can at least bring the following beneficial effects:
in the embodiment of the application, an image to be processed can be firstly used as the input of a segmentation network model, the target segmentation information of the image is determined by the segmentation network model, the image and the target segmentation information of the image are then used as the input of a matting network model, the target probability information of the image is determined by the matting network model, and finally, the designated target in the image is processed with different intensities according to the target probability information of the image, namely the probability that each pixel point belongs to the designated target.
Drawings
FIG. 1 is a flow chart of a model training method provided by an embodiment of the present application;
fig. 2 is a flowchart of an image processing method provided in an embodiment of the present application;
FIG. 3 is a schematic diagram of color transformation of a hair region of a close-up portrait according to an embodiment of the present application;
fig. 4 is a block diagram of an image processing apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
Before describing the image processing method provided by the embodiment of the present application in detail, an application scenario of the embodiment of the present application is introduced.
The method and the device for processing the designated target in the image are applied to scenes for performing special processing on the designated target in the image, for example, scenes such as performing facial beautification processing on a face in the image or performing color change on hair of the face in the image are performed. Of course, the specified target may also be set as another target, and the special processing manner may also be set as another processing manner, which may be set according to actual needs, and this is not limited in this embodiment of the application.
Next, an implementation environment related to the embodiments of the present application will be described.
The image processing method provided by the embodiment of the application can be applied to an image processing device, the image processing device can be an electronic device such as a terminal or a server, the terminal can be a mobile phone, a tablet computer or a computer, and the server can be an applied background server. As an example, an image processing application is installed in the terminal, and the terminal may execute the image processing method provided by the embodiment of the present application through the installed image processing application.
The image processing method provided in the embodiment of the application is an image processing method based on deep learning, before a specified object in an image is processed, a segmentation network model and a matting network model are needed to identify and segment the specified object in the image, the segmentation network model is used for determining object segmentation information of the image, and the matting network model is used for determining object probability information of the image according to the image and the object segmentation information of the image. Next, a model training method of the segmentation network model and the matting network model will be described.
Fig. 1 is a flowchart of a model training method provided in an embodiment of the present application, where the method is applied to an image processing apparatus, which may be an electronic device such as a terminal or a server, as shown in fig. 1, and the method includes the following steps:
step 101: target probability information of a plurality of sample images is obtained, and the target probability information of each sample image comprises the probability that each pixel point in each sample image belongs to a specified target.
The plurality of sample images are images with specified targets in the images, and the plurality of sample images meet model training requirements and are used as training samples for model training. The designated target may be preset, for example, the designated target may be a human face or hair, and of course, the designated target may also be set as another target according to actual needs, which is not limited in this embodiment of the application.
As an example, if the designated target is hair, a plurality of near view face images may be collected as a plurality of sample images for model training.
It should be noted that, because some designated objects have irregular shapes, it is difficult to accurately identify the designated objects by using an object segmentation algorithm, for example, when the designated objects are hairs, the object segmentation algorithm may only identify the approximate contours of hair regions, and cannot segment fine hairs.
As an example, the target probability information of the sample image may be presented in the form of an alpha (alpha) map, and the alpha map stores the probability that each pixel point in the sample image belongs to the specified target. For example, the alpha map may be a bitmap which stores each pixel point in the sample image in 16 bits, for example, for each pixel point in the sample image, 5 bits may represent red, 5 bits represent green, 5 bits represent blue, and the last bit is alpha, which is used to indicate the probability that the corresponding pixel point belongs to the designated target.
As an example, the target probability information of the multiple sample images may be obtained by manually labeling a specified target in the sample images by a technician, or may be obtained by identifying the images by using other target identification algorithms, and the embodiment of the present application does not limit the manner of obtaining the target probability information of the multiple sample images. For example, a technician may use image processing software to perform fine labeling on a specified target region in a sample image, so as to obtain target probability information of the sample image. The image processing software may be PS software or the like, for example.
As one example, the target probability information of the sample image may be 0 or 1. 0 is used for indicating that the corresponding pixel point does not belong to the specified target, namely belongs to the background; 1 is used to indicate that the corresponding pixel belongs to the specified target. As another example, the target probability information of the sample image may be located within an interval of [0, 1 ].
Step 102: and determining target segmentation information of each sample image according to the target probability information of each sample image in the plurality of sample images, wherein the target segmentation information comprises position information of a foreground area identified as a specified target, position information of a background area identified as a non-specified target and position information of an unknown area which cannot be identified as a background or a foreground.
That is, the target segmentation information indicates segmentation information of foreground, background, and unknown regions in the sample image, and may indicate which part of the region in the sample image belongs to the foreground, which part of the region belongs to the background, and which part of the region is a position region where it is impossible to accurately identify whether it is the foreground or the background.
As an example, the presentation form of the object segmentation information of the sample image may be a trimap (a static image matting algorithm) map in which the object segmentation information of the sample image is stored. For example, in a trimap image of a sample image, a foreground region is displayed in a first color, a background region is displayed in a second color, an unknown region is displayed in a third color, and the foreground region, the background region and a position region are distinguished by different colors.
As an example, the target probability information of each sample image may be subjected to erosion and/or dilation processing to obtain target segmentation information of each sample image. For example, erosion and/or dilation processing is performed on the alpha map of each sample image to obtain a trimap map of each sample image.
Usually, erosion and/or expansion processing is performed on the target probability information of each sample image, so that noise of each sample image can be eliminated, and fine segmentation of the foreground and the background is realized.
As another example, when determining the target segmentation information of each sample image according to the target probability information of each sample image in the plurality of sample images, for each pixel point in the sample images, if the target probability information of the pixel point is greater than or equal to a first probability threshold, the pixel point is determined as a pixel point of a foreground region, if the target probability information of the pixel point is less than the first probability threshold and greater than a second probability threshold, the pixel point is determined as a pixel point of an unknown region, and if the target probability information of the pixel point is less than the second probability threshold, the pixel point is determined as a pixel point of a background region. Wherein the first probability threshold is greater than the second probability threshold.
Wherein, the first probability threshold and the second probability threshold can be preset, for example, the first probability threshold and the second probability threshold are located in the interval of [0, 1 ]. Illustratively, the first probability threshold is 1 and the second probability threshold is 0.
Step 103: and training the segmentation network model to be trained according to the plurality of sample images and the target segmentation information of the plurality of sample images to obtain the segmentation network model.
The segmented network model to be trained and the segmented network model are deep learning network models, for example, deep learning network models such as CNN (Convolutional Neural Networks) models or RNN (Recurrent Neural Networks) models may be used. For example, the segmented network model to be trained and the segmented network model are a deep lab model (a semantic segmentation model combining CNN and probabilistic graphical model), such as deep lab v3+ (deep lab, third edition).
By training the segmentation network model to be trained according to the plurality of sample images and the target segmentation information of the plurality of sample images, the segmentation network model to be trained can continuously learn the relation between the sample images and the target segmentation information of the sample images in the training process, and then the segmentation network model capable of determining the target segmentation information of any image is obtained.
As an example, in the model training process, a plurality of sample images may be used as an input of the segmentation network model to be trained, the prediction target segmentation information of the plurality of sample images is output through the segmentation network model to be trained, then the prediction error between the prediction target segmentation information and the target segmentation information of the plurality of sample images is determined, the prediction error is propagated reversely through a back propagation algorithm to adjust the model parameters of the segmentation network model to be trained, and the segmentation network model to be trained after the model parameters are adjusted is determined as the trained segmentation network model.
As an example, the back propagation algorithm is a random gradient descent method.
Step 104: and training the matting network model to be trained according to the plurality of sample images, the target segmentation information and the target probability information of the plurality of sample images to obtain the matting network model.
The matting network model to be trained and the matting network model are deep learning network models, for example, deep learning network models such as CNN or RNN can be used. Illustratively, the training matte network model and matte network model to be trained are MBV _ DIM network models (a deep learning network model).
By training the matting network model to be trained according to the sample images, the target segmentation information of the sample images and the target probability information, the relation between the sample images, the target segmentation information of the sample images and the target probability information of the sample images can be continuously learned by the matting network model to be trained in the training process, and then the matting network model which can determine the target probability information of any image according to any image and the target segmentation information of any image is obtained.
As an example, in the model training process, a plurality of sample images and object segmentation information of the plurality of sample images may be used as inputs of a matting network model to be trained, predicted object probability information of the plurality of sample images is output through the matting network model to be trained, a prediction error between the predicted object probability information and the object probability information of the plurality of sample images is determined, the prediction error is subjected to back propagation through a back propagation algorithm to adjust model parameters of the matting network model to be trained, and the matting network model to be trained after model parameters are adjusted is determined as the well-trained matting network model.
After the training of the segmentation network model and the cutout network model is completed, the image can be processed according to the trained segmentation network model and the cutout network model. Next, the image processing procedure of the embodiment of the present application will be described in detail.
Fig. 2 is a flowchart of an image processing method provided in an embodiment of the present application, where the method is applied to an image processing apparatus, and as shown in fig. 2, the method includes the following steps:
step 201: and acquiring an image to be processed, wherein a specified target exists in the image.
The image to be processed may be a photo or an image, or may be a video frame image in a video. Moreover, the image to be processed may be uploaded by a user, acquired from a storage space of the apparatus, sent by another device, or acquired from a network, and the method for acquiring the image to be processed is not limited in the embodiment of the present application.
The designated target may be preset, for example, the designated target may be a human face or hair, and of course, the designated target may also be set as another target according to actual needs, which is not limited in this embodiment of the application.
Step 202: the image is used as an input of a segmentation network model, and target segmentation information of the image is determined through the segmentation network model, wherein the target segmentation information comprises position information of a foreground area identified as a specified target, position information of a background area identified as a non-specified target and position information of an unknown area which cannot be identified as a background or a foreground.
The segmentation network model is used for determining target segmentation information of any image. The input of the segmentation network model is an image, the output is the target segmentation information of the image, and after the image to be processed is input into the segmentation network model, the segmentation network model can output the target segmentation information of the image.
As an example, the target segmentation information of the image may be presented in the form of a trimap map, and the trimap map stores the target segmentation information of the sample image therein. For example, in a trimap image, a foreground region is displayed in a first color, a background region is displayed in a second color, an unknown region is displayed in a third color, and the foreground region, the background region and a position region are distinguished by different colors.
Step 203: the method comprises the steps of taking an image and object segmentation information of the image as input of a matting network model, determining object probability information of the image through the matting network model, wherein the object probability information of the image comprises the probability that each pixel point in the image belongs to a specified object.
The matting network model is used for determining target probability information of any image. The input of the matting network model is the image and the object segmentation information of the image, the output is the object probability information of the image, and after the image and the object segmentation information of the image are input into the matting network model, the matting network model can output the object probability information of the image.
As an example, the target probability information of the image may be presented in the form of an alpha map, and the probability that each pixel point in the image belongs to the specified target is stored in the alpha map. For example, the alpha map may be a bitmap which stores each pixel point in the image in 16 bits, for example, for each pixel point in the image, 5 bits may represent red, 5 bits may represent green, 5 bits may represent blue, and the last bit is alpha, which is used to indicate the probability that the corresponding pixel point belongs to the designated target.
Step 204: and processing the specified target in the image according to the target probability information of the image.
That is, the designated target in the image may be processed with different intensities according to the target probability information of the image, for example, for each pixel point in the image, the greater the probability that the pixel point belongs to the designated target, the greater the processing intensity of the pixel point, and the smaller the probability that the pixel point belongs to the designated target, the smaller the processing intensity of the pixel point. Thus, the specified target in the image can be finely processed.
As an example, if the processing manner is color change, the operation of processing the specified target in the image according to the target probability information of the image may include: determining a target color value to be transformed by a designated target; determining a second color value of each pixel point in the image according to the first color value of each pixel point in the image, the probability of each pixel point belonging to a specified target and the target color value; and transforming the first color value of each pixel point in the image into a second color value so as to perform color transformation on the designated target in the image.
As an example, the operation of determining the second color value of each pixel point in the image according to the first color value of each pixel point in the image, the probability that each pixel point belongs to the designated target, and the target color value includes: adding the first product and the second product to a target pixel point in the image to obtain a second color value of the target pixel point; the first product is the product between the probability that the target pixel point belongs to the designated target and the target color value, and the second product is the product between the probability that the target pixel point belongs to the background and the first color value of the target pixel point.
The probability that the target pixel belongs to the background is determined according to the probability that the target pixel belongs to the designated target, for example, the difference between 1 and the probability that the target pixel belongs to the designated target is determined as the probability that the target pixel belongs to the background.
As an example, if the image to be processed is a close-range portrait, the specified target is hair, the presentation form of the target segmentation information of the image is a trimap map, and the presentation form of the target probability information is an alpha map, then the processing procedure of performing color change on the hair region of the close-range portrait may be as shown in fig. 3. As shown in fig. 3, a close-range portrait may be used as an input of the segmentation network model, the close-range portrait may be segmented by the segmentation network model, a trimap image of the close-range portrait is output, then the close-range portrait and the trimap image of the close-range portrait are used as inputs of the matting network model, the close-range portrait is subjected to matting processing by the trimap image of the close-range portrait of the matting network model, an alpha image of the close-range portrait is obtained, and finally, fine color transformation is performed on a hair region in the close-range portrait according to the alpha image of the close-range portrait.
So, can realize the meticulous cutout effect to hair silk level in the hair region, when the user carries out the colour transform in the hair region, the colour transform of hair silk is more natural true, has improved user experience. In addition, the trimap image is generated based on the segmentation algorithm, and then fine hair-level matting is realized by adopting the matting algorithm based on the trimap image, so that the whole process can realize automatic processing, and further, the fine hair color transformation effect can be realized according to the preference of a user.
In the embodiment of the application, an image to be processed can be firstly used as the input of a segmentation network model, the target segmentation information of the image is determined by the segmentation network model, the image and the target segmentation information of the image are then used as the input of a matting network model, the target probability information of the image is determined by the matting network model, and finally, the designated target in the image is processed with different intensities according to the target probability information of the image, namely the probability that each pixel point belongs to the designated target.
Fig. 4 is a block diagram of an image processing apparatus provided in an embodiment of the present application, and as shown in fig. 4, the apparatus includes a first obtaining module 401, a segmentation module 402, a matting module 403, and a processing module 404.
A first obtaining module 401, configured to obtain an image to be processed, where the image has a specified target;
a segmentation module 402, configured to take the image as an input of a segmentation network model, determine, by the segmentation network model, target segmentation information of the image, where the target segmentation information includes location information of a foreground region identified as a designated target, location information of a background region identified as a non-designated target, and location information of an unknown region that cannot be identified as a background or a foreground, and the segmentation network model is configured to determine target segmentation information of any image;
a matting module 403, configured to use the image and the object segmentation information of the image as inputs of a matting network model, and determine object probability information of the image through the matting network model, where the object probability information of the image includes probabilities that each pixel in the image belongs to the specified object, and the matting network model is used to determine object probability information of any image;
and the processing module 404 is configured to process the specified target in the image according to the target probability information of the image.
Optionally, the processing module comprises:
a first determining unit, configured to determine a target color value to be transformed by the designated target;
the second determining unit is used for determining a second color value of each pixel point in the image according to the first color value of each pixel point in the image, the probability that each pixel point belongs to the specified target and the target color value;
and the conversion unit is used for converting the first color value of each pixel point in the image into a second color value so as to perform color conversion on the designated target in the image.
Optionally, the second determining unit is configured to:
adding the first product and the second product to a target pixel point in the image to obtain a second color value of the target pixel point;
the first product is the product between the probability that the target pixel belongs to the designated target and the target color value, the second product is the product between the probability that the target pixel belongs to the background and the first color value of the target pixel, and the probability that the target pixel belongs to the background is determined according to the probability that the target pixel belongs to the designated target.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring target probability information of a plurality of sample images;
the determining module is used for determining target segmentation information of each sample image according to the target probability information of each sample image in the plurality of sample images;
and the first training module is used for training the segmentation network model to be trained according to the plurality of sample images and the target segmentation information of the plurality of sample images to obtain the segmentation network model.
Optionally, the determining module is configured to:
and carrying out corrosion and/or expansion treatment on the target probability information of each sample image to obtain target segmentation information of each sample image.
Optionally, the apparatus further comprises:
and the second training module is used for training the matting network model to be trained according to the plurality of sample images, the target segmentation information and the target probability information of the plurality of sample images to obtain the matting network model.
Optionally, the designated object is hair or a human face.
In the embodiment of the application, an image to be processed can be firstly used as the input of a segmentation network model, the target segmentation information of the image is determined by the segmentation network model, the image and the target segmentation information of the image are then used as the input of a matting network model, the target probability information of the image is determined by the matting network model, and finally, the designated target in the image is processed with different intensities according to the target probability information of the image, namely the probability that each pixel point belongs to the designated target.
It should be noted that: in the image processing apparatus provided in the above embodiment, only the division of the functional modules is illustrated when performing image processing, and in practical applications, the functions may be distributed by different functional modules as needed, that is, the internal structure of the apparatus may be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments in detail and are not described herein again.
Fig. 5 is a schematic structural diagram of an electronic device 500 according to an embodiment of the present disclosure, where the electronic device may be a terminal or a server, and the terminal may be a mobile phone, a tablet computer, or a computer. The electronic device may have a relatively large difference due to different configurations or performances, and may include one or more processors 501 and one or more memories 502, where the memory 502 stores therein at least one instruction, and the at least one instruction is loaded and executed by the processors 501 to implement the image processing method provided by the above-mentioned method embodiments. Of course, the electronic device may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the electronic device may further include other components for implementing the functions of the device, which is not described herein again.
The embodiment of the present application further provides a computer-readable medium, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the image processing method according to the above embodiments.
The embodiment of the present application further provides a computer program product, where at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement the image processing method according to the above embodiments.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (16)
1. An image processing method, characterized in that the method comprises:
acquiring an image to be processed, wherein a specified target exists in the image;
the image is used as the input of a segmentation network model, the segmentation network model is used for determining the target segmentation information of the image, the target segmentation information comprises the position information of a foreground area identified as a specified target, the position information of a background area identified as a non-specified target and the position information of an unknown area which cannot be identified as a background or a foreground, and the segmentation network model is used for determining the target segmentation information of any image;
the image and the target segmentation information of the image are used as the input of a matting network model, the target probability information of the image is determined through the matting network model, the target probability information of the image comprises the probability that each pixel point in the image belongs to the specified target, and the matting network model is used for determining the target probability information of any image;
and processing the specified target in the image according to the target probability information of the image.
2. The method according to claim 1, wherein the processing the specified object in the image according to the object probability information of the image comprises:
determining a target color value to be transformed by the designated target;
determining a second color value of each pixel point in the image according to the first color value of each pixel point in the image, the probability of each pixel point belonging to the designated target and the target color value;
and transforming the first color value of each pixel point in the image into a second color value so as to perform color transformation on the designated target in the image.
3. The method of claim 2, wherein determining the second color value of each pixel in the image based on the first color value of each pixel in the image, the probability of each pixel belonging to the designated destination, and the destination color value comprises:
adding the first product and the second product to a target pixel point in the image to obtain a second color value of the target pixel point;
the first product is the product between the probability that the target pixel belongs to the designated target and the target color value, the second product is the product between the probability that the target pixel belongs to the background and the first color value of the target pixel, and the probability that the target pixel belongs to the background is determined according to the probability that the target pixel belongs to the designated target.
4. The method of claim 1, wherein prior to determining the target segmentation information for the image by the segmentation network model, further comprising:
acquiring target probability information of a plurality of sample images;
determining target segmentation information of each sample image according to the target probability information of each sample image in the plurality of sample images;
and training a segmentation network model to be trained according to the plurality of sample images and the target segmentation information of the plurality of sample images to obtain the segmentation network model.
5. The method of claim 4, wherein determining the target segmentation information for each sample image from the target probability information for each sample image of the plurality of sample images comprises:
and carrying out corrosion and/or expansion treatment on the target probability information of each sample image to obtain target segmentation information of each sample image.
6. The method of claim 1, wherein before determining the object probability information for the image by the matting network model, further comprising:
training a matting network model to be trained according to a plurality of sample images, and target segmentation information and target probability information of the sample images to obtain the matting network model.
7. The method of any one of claims 1-6, wherein the designated object is hair or a human face.
8. An image processing apparatus, characterized in that the apparatus comprises:
the device comprises a first acquisition module, a second acquisition module and a processing module, wherein the first acquisition module is used for acquiring an image to be processed, and a specified target exists in the image;
a segmentation module, configured to use the image as an input of a segmentation network model, determine, by the segmentation network model, target segmentation information of the image, where the target segmentation information includes location information of a foreground region identified as a designated target, location information of a background region identified as a non-designated target, and location information of an unknown region that cannot be identified as a background or a foreground, and the segmentation network model is used to determine target segmentation information of any image;
the matting module is used for taking the image and the object segmentation information of the image as input of a matting network model, determining object probability information of the image through the matting network model, wherein the object probability information of the image comprises the probability that each pixel point in the image belongs to the specified object, and the matting network model is used for determining the object probability information of any image;
and the processing module is used for processing the specified target in the image according to the target probability information of the image.
9. The apparatus of claim 8, wherein the processing module comprises:
a first determining unit, configured to determine a target color value to be transformed by the designated target;
the second determining unit is used for determining a second color value of each pixel point in the image according to the first color value of each pixel point in the image, the probability that each pixel point belongs to the specified target and the target color value;
and the conversion unit is used for converting the first color value of each pixel point in the image into a second color value so as to perform color conversion on the designated target in the image.
10. The apparatus of claim 9, wherein the second determining unit is configured to:
adding the first product and the second product to a target pixel point in the image to obtain a second color value of the target pixel point;
the first product is the product between the probability that the target pixel belongs to the designated target and the target color value, the second product is the product between the probability that the target pixel belongs to the background and the first color value of the target pixel, and the probability that the target pixel belongs to the background is determined according to the probability that the target pixel belongs to the designated target.
11. The apparatus of claim 8, further comprising:
the second acquisition module is used for acquiring target probability information of a plurality of sample images;
the determining module is used for determining target segmentation information of each sample image according to the target probability information of each sample image in the plurality of sample images;
and the first training module is used for training the segmentation network model to be trained according to the plurality of sample images and the target segmentation information of the plurality of sample images to obtain the segmentation network model.
12. The apparatus of claim 11, wherein the determining module is configured to:
and carrying out corrosion and/or expansion treatment on the target probability information of each sample image to obtain target segmentation information of each sample image.
13. The apparatus of claim 8, further comprising:
and the second training module is used for training the matting network model to be trained according to the plurality of sample images, the target segmentation information and the target probability information of the plurality of sample images to obtain the matting network model.
14. The apparatus according to any one of claims 8-13, wherein the designated object is hair or a human face.
15. An electronic device, comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the image processing method of any of claims 1 to 7.
16. A computer-readable storage medium having stored thereon at least one instruction for execution by a processor to implement the image processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911141216.9A CN110930296B (en) | 2019-11-20 | 2019-11-20 | Image processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911141216.9A CN110930296B (en) | 2019-11-20 | 2019-11-20 | Image processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110930296A true CN110930296A (en) | 2020-03-27 |
| CN110930296B CN110930296B (en) | 2023-08-08 |
Family
ID=69851170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911141216.9A Active CN110930296B (en) | 2019-11-20 | 2019-11-20 | Image processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110930296B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111462161A (en) * | 2020-03-31 | 2020-07-28 | 厦门亿联网络技术股份有限公司 | System, method, storage medium and equipment for extracting real-time video portrait |
| CN111627098A (en) * | 2020-05-21 | 2020-09-04 | 广州光锥元信息科技有限公司 | Method and device for identifying water flow area in image and generating dynamic water flow video |
| CN112819848A (en) * | 2021-02-04 | 2021-05-18 | Oppo广东移动通信有限公司 | Matting method, matting device and electronic equipment |
| CN112990331A (en) * | 2021-03-26 | 2021-06-18 | 共达地创新技术(深圳)有限公司 | Image processing method, electronic device, and storage medium |
| CN113362365A (en) * | 2021-06-17 | 2021-09-07 | 云从科技集团股份有限公司 | Video processing method, system, device and medium |
| CN113657402A (en) * | 2021-10-18 | 2021-11-16 | 北京市商汤科技开发有限公司 | Image matting processing method and device, electronic equipment and storage medium |
| WO2022227547A1 (en) * | 2021-04-29 | 2022-11-03 | 北京市商汤科技开发有限公司 | Method and apparatus for image processing, electronic device, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139415A (en) * | 2015-09-29 | 2015-12-09 | 小米科技有限责任公司 | Foreground and background segmentation method and apparatus of image, and terminal |
| CN107025457A (en) * | 2017-03-29 | 2017-08-08 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
| CN107256555A (en) * | 2017-05-25 | 2017-10-17 | 腾讯科技（上海）有限公司 | A kind of image processing method, device and storage medium |
| CN107945202A (en) * | 2017-12-19 | 2018-04-20 | 北京奇虎科技有限公司 | Image partition method, device and computing device based on adaptive threshold |
| CN108876791A (en) * | 2017-10-23 | 2018-11-23 | 北京旷视科技有限公司 | Image processing method, device and system and storage medium |
| CN108961303A (en) * | 2018-07-23 | 2018-12-07 | 北京旷视科技有限公司 | A kind of image processing method, device, electronic equipment and computer-readable medium |
| CN109461167A (en) * | 2018-11-02 | 2019-03-12 | Oppo广东移动通信有限公司 | The training method of image processing model scratches drawing method, device, medium and terminal |
| CN109658330A (en) * | 2018-12-10 | 2019-04-19 | 广州市久邦数码科技有限公司 | A kind of color development method of adjustment and device |
-
2019
- 2019-11-20 CN CN201911141216.9A patent/CN110930296B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139415A (en) * | 2015-09-29 | 2015-12-09 | 小米科技有限责任公司 | Foreground and background segmentation method and apparatus of image, and terminal |
| CN107025457A (en) * | 2017-03-29 | 2017-08-08 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
| CN107256555A (en) * | 2017-05-25 | 2017-10-17 | 腾讯科技（上海）有限公司 | A kind of image processing method, device and storage medium |
| CN108876791A (en) * | 2017-10-23 | 2018-11-23 | 北京旷视科技有限公司 | Image processing method, device and system and storage medium |
| CN107945202A (en) * | 2017-12-19 | 2018-04-20 | 北京奇虎科技有限公司 | Image partition method, device and computing device based on adaptive threshold |
| CN108961303A (en) * | 2018-07-23 | 2018-12-07 | 北京旷视科技有限公司 | A kind of image processing method, device, electronic equipment and computer-readable medium |
| CN109461167A (en) * | 2018-11-02 | 2019-03-12 | Oppo广东移动通信有限公司 | The training method of image processing model scratches drawing method, device, medium and terminal |
| CN109658330A (en) * | 2018-12-10 | 2019-04-19 | 广州市久邦数码科技有限公司 | A kind of color development method of adjustment and device |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111462161A (en) * | 2020-03-31 | 2020-07-28 | 厦门亿联网络技术股份有限公司 | System, method, storage medium and equipment for extracting real-time video portrait |
| CN111462161B (en) * | 2020-03-31 | 2023-09-26 | 厦门亿联网络技术股份有限公司 | System, method, storage medium and equipment for extracting real-time video image |
| CN111627098A (en) * | 2020-05-21 | 2020-09-04 | 广州光锥元信息科技有限公司 | Method and device for identifying water flow area in image and generating dynamic water flow video |
| CN111627098B (en) * | 2020-05-21 | 2023-04-07 | 广州光锥元信息科技有限公司 | Method and device for identifying water flow area in image and generating dynamic water flow video |
| CN112819848A (en) * | 2021-02-04 | 2021-05-18 | Oppo广东移动通信有限公司 | Matting method, matting device and electronic equipment |
| CN112819848B (en) * | 2021-02-04 | 2024-01-05 | Oppo广东移动通信有限公司 | Matting method, matting device and electronic equipment |
| CN112990331A (en) * | 2021-03-26 | 2021-06-18 | 共达地创新技术(深圳)有限公司 | Image processing method, electronic device, and storage medium |
| WO2022227547A1 (en) * | 2021-04-29 | 2022-11-03 | 北京市商汤科技开发有限公司 | Method and apparatus for image processing, electronic device, and storage medium |
| CN113362365A (en) * | 2021-06-17 | 2021-09-07 | 云从科技集团股份有限公司 | Video processing method, system, device and medium |
| CN113657402A (en) * | 2021-10-18 | 2021-11-16 | 北京市商汤科技开发有限公司 | Image matting processing method and device, electronic equipment and storage medium |
| WO2023066099A1 (en) * | 2021-10-18 | 2023-04-27 | 上海商汤智能科技有限公司 | Matting processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110930296B (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110930296B (en) | Image processing method, device, equipment and storage medium | |
| CN108229509B (en) | Method and device for identifying object class and electronic equipment | |
| CN111598091A (en) | Image recognition method and device, electronic equipment and computer readable storage medium | |
| CN108197618B (en) | Method and device for generating human face detection model | |
| CN110379020B (en) | Laser point cloud coloring method and device based on generation countermeasure network | |
| CN107292307B (en) | Automatic identification method and system for inverted Chinese character verification code | |
| CN110348358B (en) | Skin color detection system, method, medium and computing device | |
| CN111339902B (en) | Liquid crystal display indication recognition method and device for digital display instrument | |
| CN111383232A (en) | Matting method, matting device, terminal equipment and computer-readable storage medium | |
| CN111489322A (en) | Method and device for adding sky filter to static picture | |
| CN113034514A (en) | Sky region segmentation method and device, computer equipment and storage medium | |
| CN112836692A (en) | Method, apparatus, device and medium for processing image | |
| CN110852385B (en) | Image processing method, device, equipment and storage medium | |
| CN113469092A (en) | Character recognition model generation method and device, computer equipment and storage medium | |
| CN112102207A (en) | Method and device for determining temperature, electronic equipment and readable storage medium | |
| CN109241930B (en) | Method and apparatus for processing eyebrow image | |
| CN113658196A (en) | Method and device for detecting ship in infrared image, electronic equipment and medium | |
| CN110119459A (en) | Image data retrieval method and image data retrieving apparatus | |
| CN113724282A (en) | Image processing method and related product | |
| CN108491820B (en) | Method, device and equipment for identifying limb representation information in image and storage medium | |
| CN110766631A (en) | Face image modification method and device, electronic equipment and computer readable medium | |
| CN115376137A (en) | Optical character recognition processing and text recognition model training method and device | |
| CN109141457A (en) | Navigate appraisal procedure, device, computer equipment and storage medium | |
| CN114677578A (en) | Method and device for determining training sample data | |
| CN114693697A (en) | Image processing method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |