WO2022227547A1 - Method and apparatus for image processing, electronic device, and storage medium - Google Patents

Abstract

Provided in the present disclosure are a method and an apparatus for image processing, an electronic device, and a storage medium. According to an example of the method: after acquiring an original image, determining a probability value of at least some of the pixel points in the original image being target pixel points, the target pixel points being pixel points belonging to a target object in the original image; then, for any one pixel point amongst the at least some pixel points, on the basis of the probability value of the pixel point being a target pixel point and a pre-selected target colour value, determining a fusion colour value corresponding to the pixel point; and finally, using the fusion colour values corresponding to the at least some pixel points to adjust the colour values of the at least some pixel points in the original image in order to obtain a target image.

Description

Method, apparatus, electronic device and storage medium for image processing

Cross-reference to related applications

This disclosure claims the priority of the Chinese patent application filed on April 29, 2021 with the application number 202110473454.0 and the invention titled "image processing method, device, electronic device and storage medium", the entire content of which is disclosed in the Chinese patent application Incorporated herein by reference.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to a method, an apparatus, an electronic device, and a storage medium for image processing.

Background technique

As a conventional information-bearing medium, images are widely used in various scenarios such as work and life. By modifying the image parameters of the pixels in the image, the picture in the image can be modified. For example, you can change the color value of an object in the picture by replacing the color value of the pixels in the image. However, the color of the picture is usually more complicated. If the color value of the pixel of an object in the picture is directly replaced with a certain value, the part of the object connected with other parts in the picture will lose the transition color, which will make the There is a big difference between the effect of the object after the color change and the original image.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide at least an image processing method, an apparatus, an electronic device, and a storage medium.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including: acquiring a probability value that at least some pixels in an original image are target pixels, wherein the target pixels are in the original image and belong to a target object The pixel point; for any pixel point in the at least part of the pixel point, based on the probability value that the pixel point is the target pixel point and the preselected target color value, determine the fusion color value corresponding to the pixel point ; Adjust the color values of the at least part of the pixel points in the original image by using the fusion color values corresponding to the at least part of the pixel points respectively, to obtain a target image.

In this aspect, by determining the probability value that at least some of the pixels in the original image are target pixels, the target object whose color needs to be replaced can be identified from the original image. Determine the fusion color value corresponding to at least part of the pixel points based on the probability value that at least part of the pixel points are the target pixel point and the preselected target color value, and then use the fusion color value to adjust the color value of at least part of the pixel points to obtain the target image, In the target image, there is a certain difference between the pixels near and inside the target object to form a transition color, so that the characteristics of the target object after color replacement are more in line with the original image.

In a possible implementation manner, the obtaining the probability value that at least some of the pixels in the original image are target pixels includes: extracting image feature information in the original image; determining the image feature information based on the image feature information At least some of the pixels in the original image are the initial probability values of the target pixels; based on the initial probability values of the at least some of the pixels as the target pixels and the color values of the at least some of the pixels, the standard color value is determined ; For any pixel point in the at least part of the pixel points, based on the standard color value and the color value of the pixel point, the initial probability value that the pixel point is the target pixel point is calibrated to obtain the The pixel point is the probability value of the target pixel point.

In this embodiment, a standard color value is determined based on the initial probability value and color value of at least some of the pixels for the target pixel, and then the initial probability value is corrected by using the color values and standard color values corresponding to at least some of the pixels, so that at least The probability value that some of the pixels are the target pixels is more in line with the real situation, and the accuracy of identifying the target object is improved.

In a possible implementation manner, the determining, based on the image feature information corresponding to the original image, that at least some of the pixels in the original image are the initial probability values of the target pixel points includes: based on the image features information, determine that at least some of the pixels in the original image are the evaluation scores of the target pixels; for any pixel in the at least some of the pixels, based on the preset evaluation score correction threshold and the pixel are The evaluation score of the target pixel point determines that the pixel point is the initial probability value of the target pixel point.

In this embodiment, the evaluation scores of at least some of the pixels are modified by using the evaluation score correction threshold to obtain the initial probability value, so that the initial probability value is more accurate.

In a possible implementation manner, the evaluation score correction threshold includes a first correction threshold and a second correction threshold, and the first correction threshold is greater than the second correction threshold. The determining that the pixel point is an initial probability value of the target pixel point based on the preset evaluation score correction threshold and the pixel point being the evaluation score of the target pixel point includes: when the pixel point is the target pixel point. When the evaluation score of the target pixel is greater than or equal to the first correction threshold, determine that the pixel is the initial probability value of the target pixel. The initial probability value is the first preset probability value; In the case where the evaluation score of the target pixel is less than or equal to the second correction threshold, determine that the pixel is the initial probability value of the target pixel as the second preset probability value; In the case where the evaluation score of the target pixel is less than the first correction threshold and greater than the second correction threshold, the pixel is based on the evaluation score of the target pixel, the first correction threshold and the second correction threshold. The modified threshold determines the initial probability value that the pixel point is the target pixel point.

In this embodiment, by setting the first correction threshold and the second correction threshold, the eligible pixels can be directly determined as the initial probability value of the target pixel, which can reduce the amount of calculation for calculating the initial probability value and improve the calculation efficiency.

In a possible implementation manner, the pixel point is determined as the target pixel point based on the evaluation score of the pixel point as the target pixel point, the first correction threshold and the second correction threshold The initial probability value of , including: determining the first difference between the first correction threshold and the second correction threshold; determining that the pixel point is the evaluation score of the target pixel point and the second correction threshold The second difference between the two; based on the ratio between the second difference and the first difference, determine that the pixel is an initial probability value of the target pixel.

In this embodiment, the initial probability value is determined by using the ratio between the second difference value and the first difference value, which can improve the accuracy of the initial probability value.

In a possible implementation manner, the determining a standard color value based on the initial probability value that the at least part of the pixel points are the target pixel point and the color value of the at least part of the pixel point includes: determining the at least part of the pixel point. In some pixel points, the initial probability value is higher than or equal to the first pixel point with a preset probability threshold; the average value of the color values of all the first pixel points is used as the standard color value.

In a possible implementation manner, the initial probability value that the pixel point is the target pixel point is calibrated based on the standard color value and the color value of the pixel point, and the pixel point is obtained as The probability value of the target pixel point includes: determining the similarity between the color value of the pixel point and the standard color value; based on the similarity, determining the initial probability that the pixel point is the target pixel point The value is calibrated to obtain the probability value that the pixel point is the target pixel point.

In this embodiment, the initial probability is corrected by using the similarity between the color value of the pixel point and the standard color value, so as to further improve the accuracy of the probability value.

In a possible implementation manner, the determining the fusion color value corresponding to the pixel point based on the probability value that the pixel point is the target pixel point and the preselected target color value includes: based on the pixel point The point is the probability value of the target pixel point, and the color fusion weight corresponding to the pixel point is determined; based on the color value of the pixel point, the target color value and the color fusion weight corresponding to the pixel point, the color fusion weight corresponding to the pixel point is determined The fusion color value corresponding to the pixel point.

In this embodiment, the color fusion weight corresponding to the pixel point is determined based on the probability value that the pixel point is the target pixel point, and then the color value of the pixel point and the target color value are fused based on the color fusion weight to obtain the corresponding pixel point. Fusion color value, so that the fusion color value can reflect the original color value of the pixel and the target color value according to the color fusion weight, so that the boundary of the target object forms a transition color, so that the characteristics of the target object after color replacement are more in line with the original image. .

In a possible implementation manner, after adjusting the color values of the at least part of the pixel points by using the determined fusion color value to obtain the target image, the method further includes: for the at least part of the pixel points any pixel point of the original image, determine the target gray-scale value corresponding to the pixel point based on the gray-scale value of the pixel point in the original image and/or the preset gray-scale value corresponding to the pixel point; The grayscale value of the pixel point corresponding to the pixel point in the target image is adjusted to the target grayscale value corresponding to the pixel point.

In this embodiment, the target gray-scale value of at least some pixels in the target image is determined by using the gray-scale values of at least some pixels in the original image and/or the preset gray-scale values of at least some pixels. The brightness of the target object is adjusted so that the characteristics of the target object after color replacement are more in line with user expectations.

In a possible implementation manner, the extracting image feature information in the original image includes: extracting image feature information of the original image at multiple preset resolutions. The determining, based on the image feature information, that at least some of the pixels in the original image are the evaluation scores of the target pixels, includes: using a preset classifier to determine the lowest resolution among the multiple preset resolutions At least some of the pixels in the original image are the initial evaluation scores of the target pixels under the preset resolution of At least some of the pixels in the original image are the initial evaluation scores of the target pixels and the image feature information of the original image at the current preset resolution, and determine the original image at the current preset resolution. At least some of the pixels in the image are the intermediate evaluation scores of the target pixels; at least some of the pixels in the original image will be the target pixels at the highest preset resolution among the multiple preset resolutions The intermediate evaluation score of , as the evaluation score of at least some of the pixels in the original image being the target pixel.

In this embodiment, by determining the initial evaluation scores of the original image at multiple preset resolutions, and then determining the evaluation scores of at least some of the pixels as target pixels based on the initial evaluation scores, the accuracy of calculating the probability value can be improved.

In a possible implementation manner, the original image includes a live image captured by an augmented reality AR device, and the AR device displays the target image.

In this embodiment, the original image is acquired by the AR device, and the target image is displayed by the AR device, so that the real-time color change of the target object in the AR scene can be realized.

In a possible implementation manner, the original image includes: an image of a target person, wherein the target object includes at least one of a human hair area, a human skin area, and at least part of a clothing area in the target person image or a target object image, wherein the target object is at least part of the object area in the target object image.

In this embodiment, the target person image or the target object image is used as the original image, so as to realize the color adjustment of the human hair area, the human skin area, the clothing area or the object area.

In a second aspect, an embodiment of the present disclosure further provides an image processing apparatus, including: an acquisition module configured to acquire a probability value that at least some of the pixels in the original image are target pixels, wherein the target pixels are the original A pixel point belonging to the target object in the image; a determination module, configured to determine, for any pixel point in the at least part of the pixel point, based on the probability value that the pixel point is the target pixel point and the preselected target color value The fusion color value corresponding to the pixel point; the generating module is configured to adjust the color value of the at least part of the pixel point in the original image by using the fusion color value corresponding to the at least part of the pixel point respectively, to obtain the target image.

In a third aspect, embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions; the processor is configured to execute the machine-readable instructions stored in the memory , wherein, when the electronic device is running, the processor and the memory communicate through a bus, and the machine-readable instructions are executed by the processor to execute the first aspect or any one of the first aspects. steps in a possible implementation.

In a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the first aspect, or any one of the first aspect. steps in one possible implementation.

For a description of the effects of the above image processing apparatus, electronic equipment, and computer-readable storage medium, reference may be made to the description of the above image processing method, and details are not repeated here.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. These drawings illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of a confidence map provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of a neural network provided by an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. The described embodiments are only some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this paper only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: the existence of A alone, the existence of A and B at the same time, the existence of B alone. a situation. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

After research, it is found that when performing color-changing processing on some objects in the image, if the color value of the pixel of an object in the picture is directly replaced with a specific value, the part of the object connected to other parts in the picture will be changed. Losing the transition color makes the effect of the object after the color change appear quite different from the original image.

Based on the above research, the present disclosure provides a method, an apparatus, an electronic device and a storage medium for image processing. By determining the probability value that at least some of the pixels in the original image are target pixels, it is possible to identify the color that needs to be replaced from the original image. The target object, the fusion color value corresponding to at least part of the pixel points is determined based on the probability value of at least part of the pixel points for the target pixel point and the preselected target color value, and then the target image is generated based on the fusion color value. In this way, in the target image, there is a certain difference between the pixels near and inside the target object, forming a transition color, so that the characteristics of the target object after color replacement are more in line with the original image.

The above are all obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions to the above problems proposed by the present disclosure below should be made by the inventor in the process of the present disclosure. contribution.

The method, apparatus, electronic device, and computer-readable storage medium for image processing disclosed in the present disclosure will be described below through specific embodiments.

As shown in FIG. 1 , an embodiment of the present disclosure discloses an image processing method, which can be applied to an electronic device with computing capability, such as a server. Specifically, the image processing method may include the following steps:

S110. Obtain a probability value that at least some of the pixels in the original image are target pixels. Wherein, the target pixel point is a pixel point belonging to the target object in the original image.

In some possible implementations, the above-mentioned original image may be a live image captured by an augmented reality (AR) (Augmented Reality) device. When the AR device captures a live image, the original image can be acquired in real time, and the probability value that at least some of the pixels in the original image are target pixels can be determined. The AR device may be a smart terminal with AR function held by the user, and may include, but is not limited to, electronic devices such as mobile phones, tablet computers, and AR glasses that can present augmented reality effects. The above-mentioned at least some of the pixels may be all pixels in the original image, or may be pixels in a preset area, and the preset area may be an area pre-selected by a user.

The target pixel can be a pixel belonging to the target object in the original image, and the target object can be the area where the object that needs to be replaced by color is located.

Exemplarily, the original image may be an image containing hair, and the hair may specifically be human hair, animal hair, or the like. The area that needs to be replaced by color is, for example, the hair area of a human body, or the hair area of an animal.

Exemplarily, the original image may be a target person image, and the target object may include, but is not limited to, at least one of a human hair area, a human skin area, a human eye area, and at least part of a clothing area in the target person image.

Further, the original image may also be an image of a target object, and the target object image may include one or more object objects. At this time, the target object may be at least part of the object area in the target object image. The object may be a tree area in the target object image, and the tree area may be a trunk area of a tree, a leaf area, or an entire area of the tree.

In some possible embodiments, the probability value that at least some of the pixels in the original image are target pixels may be determined by the following steps: extracting image feature information in the original image; determining the original image based on the image feature information At least some of the pixels in the image are the initial probability values of the target pixels; based on the initial probability values of the at least some of the pixels as the target pixels and the color values of the at least some of the pixels, a standard color value is determined; Based on the standard color value and the color value of the at least part of the pixel points, the initial probability value of the at least part of the pixel points being the target pixel point is calibrated, and the calibrated initial probability value is used as the The probability value that at least some of the pixels in the original image are the target pixels.

The above-mentioned determining that at least some of the pixels in the original image are the initial probability values of the target pixels may include: determining, based on the image feature information, that at least some of the pixels in the original image are the evaluation scores of the target pixels; For any pixel in the at least part of the pixels, based on the preset evaluation score correction threshold and the evaluation score of the pixel as the target pixel, determine the initial probability value of the pixel as the target pixel.

The above image feature information may be feature parameters of at least some pixels in the original image, such as color value, saturation, brightness, and the like. The trained neural network can be used to process the image feature information to obtain the evaluation score that at least some of the pixels are the target pixel, and then use the evaluation score to correct the threshold and the evaluation score to determine that at least some of the pixels in the original image are the target pixel. initial probability value.

In some possible embodiments, image feature information of the original image at various preset resolutions may be extracted, and then the trained neural network may be used to determine the evaluation scores of at least some of the pixels as target pixels.

Exemplarily, as shown in FIG. 3 , which is a schematic diagram of a neural network provided by an embodiment of the present disclosure, the neural network may use a feature extractor to extract image feature information of an original image at multiple preset resolutions. Specifically, the image feature information of the original image at the highest preset resolution can be extracted first, and then the obtained image feature information can be down-sampled to obtain the image feature information of the original image at each preset resolution.

After obtaining the image feature information of the original image at each preset resolution, the neural network can use the preset classifier to determine the initial evaluation score that at least some of the pixels in the original image are target pixels at the lowest preset resolution . Then, in the order of the preset resolution from low to high, based on the initial evaluation score of at least some of the pixels in the original image at the previous preset resolution as target pixels, and the image features of the original image at the current preset resolution information and a classifier to determine that at least some of the pixels in the original image at the current preset resolution are the intermediate evaluation scores of the target pixels. Finally, at the highest preset resolution, at least some of the pixels in the original image are the intermediate evaluation scores of the target pixels, and output as the evaluation scores of at least some of the pixels in the original image as the target pixels.

Exemplarily, after the initial evaluation score under the current preset resolution is obtained, the initial evaluation score under the previous preset resolution may be up-sampled, so that the resolution of the evaluation score under the previous preset resolution is the same as that of the previous preset resolution. The resolutions of the evaluation scores under the current preset resolution are the same, and then the evaluation scores under the two preset resolutions are spliced to obtain the spliced evaluation scores under the current preset resolution.

The above neural network can be trained by using a pre-prepared data set. The data set can include a large number of images and labels corresponding to the images. The images contain target objects, and the labels can be marked with pixels corresponding to the target objects. Using the neural network shown in Figure 3, the output of different preset resolutions can be supervised, and the small-resolution output can be fused with the features of the low-level large-resolution output, so that the small-resolution output can learn more Good semantic categories, while the large-resolution output learns finer details, thereby improving the accuracy of the neural network. After the dataset is input to the neural network, the loss can be calculated according to the output results of the neural network and the labeling results. Compared with other backgrounds, the target object often has a color jump at the boundary, so it has a clear gradient. Therefore, a refine loss loss function that can better preserve the boundary gradient can be used, so that the boundary of the target object in the image output by the neural network has a similar gradient to the image input to the neural network, so as to obtain a more refined edge prediction result.

In this embodiment, the pixel points can be classified by using the evaluation score correction threshold, so that different initial probability value calculation methods can be adopted for different types of pixel points, so as to facilitate calculation optimization.

In some possible embodiments, the evaluation score modification threshold includes a first modification threshold and a second modification threshold; the first modification threshold is greater than the second modification threshold. Correspondingly, determining the pixel point as the initial probability value of the target pixel point based on the preset evaluation score correction threshold and the pixel point as the evaluation score of the target pixel point, including: evaluating the pixel point When the score is greater than or equal to the first correction threshold, the initial probability value of the pixel as the target pixel is determined to be the first preset probability value; the evaluation score of the pixel is less than or equal to the second In the case of correcting the threshold, the initial probability value of determining that the pixel is the target pixel is a second preset probability value; the evaluation score of the pixel is less than the first correction threshold and greater than the second correction threshold In the case of , the initial probability value that the pixel is the target pixel is determined based on the evaluation score of the pixel being the target pixel, the first correction threshold and the second correction threshold.

Exemplarily, the first correction threshold may be set to 0.7, and the second correction threshold may be set to 0.4. For a pixel whose evaluation score is less than or equal to the second correction threshold, the probability of the pixel being the target pixel is low and can be ignored, so the initial probability value of the pixel being the target pixel can be directly set to 0, that is The second preset probability value. Correspondingly, for a pixel whose evaluation score is greater than or equal to the first correction threshold, the probability of the pixel being the target pixel is relatively high, so the initial probability value of the pixel being the target pixel can be set to 1, that is, the first Preset probability value.

In this way, by first setting the initial probability value of the pixel point with the evaluation score high enough (greater than the first correction threshold) and low enough (less than the second correction threshold) as the target pixel point as the corresponding preset probability value, and then for The initial probability value calculation is performed on the pixel points whose evaluation score is greater than the second correction threshold and less than the first correction threshold, which can greatly reduce the calculation time and improve the calculation efficiency.

The above-mentioned determining that the pixel is the initial probability value of the target pixel based on the evaluation score of the pixel as the target pixel, the first correction threshold and the second correction threshold may include: determining the first correction threshold. a first difference between a modified threshold and the second modified threshold; determining a second difference between the evaluation score and the second modified threshold; based on the second difference and the first The ratio between the differences determines the initial probability value.

Specifically, the ratio between the second difference and the first difference may be used as the initial probability value.

For a pixel whose initial probability value is higher than or equal to the preset probability threshold, the pixel can be considered as a target pixel. In addition, the average value of the color values of the plurality of pixel points whose initial probability value is higher than or equal to the preset probability threshold value can be used as the standard color value, and the initial probability value can be corrected by using the standard color value, so as to obtain at least some of the pixel points as the target The probability value of the pixel point.

The preset probability threshold may be the same as the first correction threshold, and the mean value of the color values of the pixels whose initial probability value is higher than or equal to the preset probability threshold may be determined, and the mean value may be used as the standard color value. In this way, since the standard color value is relatively close to the color value of the target pixel point, the standard color value can be used to correct the initial probability value of each pixel point being the target pixel point.

In some possible embodiments, the similarity between the color value of the pixel and the standard color value may be determined for any of the above at least some of the pixels; The point is the initial probability value of the target pixel for calibration. Exemplarily, the initial probability value may be weighted by using the similarity between the color value of the pixel point and the standard color value, and the weighted probability value may be used as the probability value that the pixel point is the target pixel point.

Exemplarily, as shown in FIG. 2 , it is a schematic diagram of a confidence map provided by an embodiment of the present disclosure. The above confidence map can represent the probability value that at least some of the pixels in the original image are target pixels. The original image corresponding to the confidence map shown in FIG. 2 is a human image, and the target object is the area where human hair is located. Obviously, the probability value of the target pixel corresponding to the hair in Figure 2 is close to 1, which tends to be white in the confidence map, while the probability value of the pixels corresponding to other parts is close to 0, which tends to be in the confidence map. black.

S120. For any pixel point in the at least part of the pixel points, based on the probability value that the pixel point is the target pixel point and the preselected target color value, determine the fusion color value corresponding to the pixel point.

The above-mentioned target color value may be a color value preset by a user. For example, if the user wishes to replace the hair color in Figure 2 with brown, the color value corresponding to brown can be set as the target color value.

In some possible embodiments, determining the corresponding fusion color value based on the probability value that the pixel point is the target pixel point and the preselected target color value may include: based on the probability that the pixel point is the target pixel point. value, and determine the color fusion weight corresponding to the pixel point; based on the color value of the pixel point, the target color value, and the color fusion weight corresponding to the pixel point, determine the fusion color value corresponding to the pixel point.

Exemplarily, the original color value of the pixel point (that is, the color value of the pixel point in the original image, which may also be referred to as the color value of the pixel point) and the fusion weight of the target color value can be determined based on the above probability value, and According to the ratio of the fusion weight, the original color value of the pixel point and the target color value are fused to obtain the fusion color value corresponding to the pixel point.

Specifically, the original color value of the pixel point and the target color value can be fused by means of alpha fusion.

In this embodiment, by first determining the color fusion weight corresponding to the pixel point based on the probability value that the pixel point is the target pixel point, and then fusing the color value of the pixel point and the target color value based on the color fusion weight, the result is obtained. The fusion color value corresponding to the pixel point, so that the fusion color value can reflect the original color value of the pixel point and the target color value according to the color fusion weight, so that the boundary of the target object can form a transition color, so that the target object can be replaced after color replacement. The features are more in line with the original image.

S130. Adjust the color values of the at least some of the pixels in the original image by using the fusion color values corresponding to the at least some of the pixels to obtain a target image.

In this step, the color values of at least some of the pixels in the original image may be replaced according to the fusion color values corresponding to the above at least some of the pixels to obtain the target image.

After the target image is generated, based on the grayscale values of the at least part of the pixels in the original image, and/or the preset grayscale values of the at least part of the pixels, it is possible to determine, respectively, corresponding to the at least part of the pixels. target grayscale value, and then adjust the grayscale value of the at least part of the pixel points in the target image to the corresponding target grayscale value.

In this way, the brightness of the target object after the color change can be adjusted, so that the target object can keep the original texture and brightness after the color replacement is performed, and the characteristics are more in line with the user's expectation.

After the target image is obtained, an AR device can also be used to display the obtained target image, so as to realize the real-time color change of the target object in the AR scene.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, the embodiment of the present disclosure also provides an image processing apparatus corresponding to the image processing method. Reference can be made to the implementation of the method, and repeated details will not be repeated.

Referring to FIG. 4 , which is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure, the apparatus includes: an acquisition module 410 for acquiring a probability value that at least some of the pixels in the original image are target pixels. Wherein, the target pixel point is a pixel point belonging to the target object in the original image.

The determining module 420 is configured to, for any pixel point in the at least part of the pixel points, determine the fusion color value corresponding to the pixel point based on the probability value that the pixel point is the target pixel point and the preselected target color value.

The generating module 430 is configured to adjust the color values of the at least part of the pixel points in the original image by using the fusion color values corresponding to the at least part of the pixel points respectively, to obtain a target image.

In a possible implementation manner, the obtaining module 410 is specifically configured to: extract image feature information in the original image; determine at least some of the pixels in the original image as the target based on the image feature information the initial probability value of the pixel point; based on the initial probability value of the at least part of the pixel point being the target pixel point and the color value of the at least part of the pixel point, determine a standard color value; based on the standard color value and the at least color values of some pixels, calibrate the initial probability value that at least some of the pixels are the target pixel, and use the calibrated initial probability value as at least some of the pixels in the original image. The probability value of the target pixel point.

In a possible implementation manner, when determining, based on the image feature information, that at least some of the pixels in the original image are the initial probability values of the target pixels, the obtaining module 410 is specifically configured to: the image feature information, determine that at least some of the pixels in the original image are the evaluation scores of the target pixels; for any pixel in the at least part of the pixels, based on the preset evaluation score correction threshold and the pixel are The evaluation score of the target pixel point determines that the pixel point is the initial probability value of the target pixel point.

In a possible implementation, the evaluation score modification threshold includes a first modification threshold and a second modification threshold; the first modification threshold is greater than the second modification threshold.

When the acquisition module 410 determines that the pixel is the initial probability value of the target pixel based on the preset evaluation score correction threshold and the pixel is the evaluation score of the target pixel, it is specifically used for: In the case that the evaluation score of the pixel point is greater than or equal to the first correction threshold, determine that the pixel point is the initial probability value of the target pixel point as the first preset probability value; when the evaluation score of the pixel point is less than or equal to the second correction threshold, determine that the initial probability value of the pixel point as the target pixel point is a second preset probability value; the evaluation score of the pixel point is less than the first correction threshold value and In the case of being greater than the second correction threshold, based on the evaluation score of the pixel point being the target pixel point, the first correction threshold value and the second correction threshold value, it is determined that the pixel point is a part of the target pixel point. initial probability value.

In a possible implementation manner, the obtaining module 410 determines that the pixel is the target based on the evaluation score of the pixel as the target pixel, the first correction threshold and the second correction threshold When the initial probability value of the pixel point is used, it is specifically used to: determine the first difference between the first correction threshold and the second correction threshold; determine the first difference between the evaluation score and the second correction threshold Two difference values; the initial probability value is determined based on the ratio between the second difference value and the first difference value.

In a possible implementation manner, when the obtaining module 410 determines the standard color value based on the initial probability value that the at least part of the pixel points are the target pixel point and the color value of the at least part of the pixel point, the specific It is used for: determining the mean value of the color values of a plurality of pixel points whose initial probability value is higher than or equal to a preset probability threshold value, and using the mean value as the standard color value.

When calibrating the initial probability value that the at least part of the pixel points are the target pixel point based on the standard color value and the color value of the at least part of the pixel points, the obtaining module 410 is specifically configured to: any pixel point in the at least part of the pixel points, to determine the similarity between the color value of the pixel point and the standard color value; based on the similarity, the initial probability that the pixel point is the target pixel point value to calibrate.

In a possible implementation manner, when the determining module 420 determines the corresponding fusion color value based on the probability value that the at least part of the pixel points are the target pixel point and the preselected target color value, it is specifically used for : Determine the color fusion weight corresponding to the pixel point based on the probability value that the pixel point is the target pixel point; based on the color value of the pixel point, the target color value and the color fusion corresponding to the pixel point Weight, to determine the fusion color value corresponding to the pixel point.

In a possible implementation manner, the generating module 430 is further configured to: based on the grayscale values of the at least part of the pixels in the original image and/or the preset grayscale values of the at least part of the pixels , determine the target grayscale values corresponding to the at least part of the pixel points respectively; adjust the grayscale values of the pixel points corresponding to the at least part of the pixel points in the target image to the target grayscale value corresponding to the pixel points order value.

In a possible implementation manner, when extracting the image feature information in the original image, the obtaining module 410 is specifically configured to: extract image feature information of the original image under multiple preset resolutions.

When determining, based on the image feature information, that the at least some of the pixels are the evaluation scores of the target pixels, the obtaining module is specifically configured to: use a preset classifier to determine, by using a preset classifier, that in the original image at the lowest preset resolution. At least some of the pixels are the initial evaluation scores of the target pixels; according to the preset resolution from low to high, based on the previous preset resolution, at least some of the pixels in the original image are the initial evaluation scores of the target pixels and the original The image feature information of the image at the current preset resolution, determining that at least some of the pixels in the original image under the current preset resolution are the intermediate evaluation scores of the target pixel; the original image at the highest preset resolution is determined. The intermediate evaluation scores of at least some of the pixels in the original image are taken as the evaluation scores of at least some of the pixels in the original image being the target pixels.

In a possible implementation manner, when acquiring the probability value that at least some of the pixels in the original image are target pixels, the acquiring module 410 is specifically configured to: take the live image captured by the augmented reality AR device as the original image, and acquire The probability value that at least some of the pixels in the original image are the target pixels.

The generating module 430 is further configured to: display the target image by using the AR device.

In a possible implementation manner, when acquiring the probability value that at least some of the pixels in the original image are target pixels, the acquiring module 410 is specifically configured to: take the target person image as the original image, wherein the target object Including at least one of the human hair area, human skin area, and at least part of the clothing area in the target person image; or, using the target object image as the original image, wherein the target object is in the target object image. at least part of the object area. Obtain a probability value that at least some of the pixels in the original image are target pixels.

Corresponding to the image processing method in FIG. 1 , an embodiment of the present disclosure further provides an electronic device 500 . As shown in FIG. 5 , a schematic structural diagram of the electronic device 500 provided by the embodiment of the present disclosure includes: a processor 51 and a memory 52 , and bus 53. The memory 52 is used to store execution instructions, including a memory 521 and an external memory 522, where the memory 521 is also called an internal memory, and is used to temporarily store operation data in the processor 51 and data exchanged with an external memory 522 such as a hard disk. The processor 51 exchanges data with the external memory 522 through the memory 521. When the electronic device 500 is running, the processor 51 communicates with the memory 52 through the bus 53, so that the processor 51 can execute the following instructions :

Obtain the probability value that at least some of the pixels in the original image are target pixels, where the target pixels are pixels belonging to the target object in the original image; for any pixel in the at least some of the pixels, Based on the probability value that the pixel point is the target pixel point and the preselected target color value, determine the fusion color value corresponding to the pixel point; use the fusion color value corresponding to the at least part of the pixel points to compare the original The color values of the at least part of the pixels in the image are adjusted to obtain the target image.

For the specific execution process of the above instruction, reference may be made to the steps of the image processing method described in the embodiments of the present disclosure, and details are not described herein again.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the image processing method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure further provide a computer program product, including a computer-readable storage medium storing program codes, wherein the instructions included in the program codes can be used to execute the steps of the image processing methods in the above method embodiments, specifically Refer to the above method embodiments, which are not repeated here.

Wherein, the computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, but not to limit them. The protection scope of the present disclosure is not limited to this, although the aforementioned The embodiments describe the present disclosure in detail, and those skilled in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (15)

1. An image processing method, comprising:

   Obtain the probability value that at least some of the pixels in the original image are target pixels, wherein the target pixels are pixels belonging to the target object in the original image;

   For any pixel point in the at least part of the pixel points, based on the probability value that the pixel point is the target pixel point and the pre-selected target color value, determine the fusion color value corresponding to the pixel point;

   A target image is obtained by adjusting the color values of the at least part of the pixel points in the original image by using the fusion color values corresponding to the at least part of the pixel points respectively.
2. The method according to claim 1, wherein the acquiring the probability value that at least some of the pixels in the original image are the target pixels comprises:

   extracting image feature information in the original image;

   Based on the image feature information, determining that at least some of the pixels in the original image are the initial probability values of the target pixels;

   determining a standard color value based on the initial probability value of the at least part of the pixel points being the target pixel point and the color value of the at least part of the pixel point;

   For any pixel point in the at least part of the pixel points, based on the standard color value and the color value of the pixel point, the initial probability value that the pixel point is the target pixel point is calibrated to obtain the pixel point The point is the probability value of the target pixel point.
3. The method according to claim 2, wherein the determining, based on the image feature information, that at least some of the pixels in the original image are the initial probability values of the target pixels, comprising:

   Based on the image feature information, determining that at least some of the pixels in the original image are the evaluation scores of the target pixels;

   For any pixel point in the at least some of the pixel points, based on the preset evaluation score correction threshold and the evaluation score of the pixel point as the target pixel point, determine the initial probability that the pixel point is the target pixel point value.
4. The method according to claim 3, wherein the evaluation score correction threshold comprises a first correction threshold and a second correction threshold, the first correction threshold is greater than the second correction threshold; The evaluation score correction threshold and the evaluation score of the pixel point as the target pixel point, and determining that the pixel point is the initial probability value of the target pixel point, including:

   In the case where the evaluation score of the pixel point being the target pixel point is greater than or equal to the first correction threshold, determining that the pixel point is the initial probability value of the target pixel point as a first preset probability value;

   In the case that the evaluation score of the pixel point being the target pixel point is less than or equal to the second correction threshold, determining the initial probability value of the pixel point being the target pixel point as a second preset probability value;

   In the case where the evaluation score of the pixel point being the target pixel point is less than the first correction threshold and greater than the second correction threshold, based on the evaluation score of the pixel point being the target pixel point, the first correction threshold The correction threshold and the second correction threshold determine the initial probability value that the pixel point is the target pixel point.
5. The method according to claim 4, wherein the pixel is determined to be the target pixel based on the evaluation score of the pixel as the target pixel, the first correction threshold and the second correction threshold. Describe the initial probability value of the target pixel, including:

   determining a first difference between the first modified threshold and the second modified threshold;

   determining that the pixel point is the second difference between the evaluation score of the target pixel point and the second correction threshold;

   Based on the ratio between the second difference value and the first difference value, the initial probability value of the pixel point as the target pixel point is determined.
6. The method according to any one of claims 2 to 5, wherein the standard color is determined based on an initial probability value that the at least part of the pixel points are the target pixel point and a color value of the at least part of the pixel point values, including:

   determining a first pixel whose initial probability value is higher than or equal to a preset probability threshold in the at least part of the pixel points;

   The average value of the color values of all the first pixel points is used as the standard color value.
7. The method according to any one of claims 2 to 6, characterized in that, based on the standard color value and the color value of the pixel point, performing an initial probability value on the pixel point being the target pixel point. Calibration to obtain the probability value that the pixel point is the target pixel point, including:

   determining the similarity between the color value of the pixel and the standard color value;

   The initial probability value that the pixel point is the target pixel point is calibrated based on the similarity, to obtain the probability value that the pixel point is the target pixel point.
8. The method according to any one of claims 1 to 7, wherein the fusion color corresponding to the pixel point is determined based on a probability value that the pixel point is the target pixel point and a preselected target color value values, including:

   Determine the color fusion weight corresponding to the pixel point based on the probability value that the pixel point is the target pixel point;

   Based on the color value of the pixel point, the target color value and the color fusion weight corresponding to the pixel point, the fusion color value corresponding to the pixel point is determined.
9. The method according to any one of claims 1 to 8, wherein the method further comprises:

   For any pixel point in the at least part of the pixel points,

   Determine the target grayscale value corresponding to the pixel point based on the grayscale value of the pixel point in the original image and/or the preset grayscale value corresponding to the pixel point;

   Adjust the grayscale value of the pixel point corresponding to the pixel point in the target image to the target grayscale value corresponding to the pixel point.
10. The method of claim 3, wherein:

    The extracting image feature information in the original image includes:

    extracting image feature information of the original image at multiple preset resolutions;

    The determining, based on the image feature information, that at least some of the pixels in the original image are the evaluation scores of the target pixels, including:

    Using a preset classifier, determining that at least some of the pixels in the original image at the lowest preset resolution among the multiple preset resolutions are the initial evaluation scores of the target pixels;

    According to the order of the plurality of preset resolutions from low to high, based on the initial evaluation score and the Image feature information of the original image at the current preset resolution, determining that at least some of the pixels in the original image under the current preset resolution are the intermediate evaluation scores of the target pixels;

    Taking at least some of the pixels in the original image at the highest preset resolution among the multiple preset resolutions as the intermediate evaluation score of the target pixel, as at least some of the pixels in the original image Describe the evaluation score of the target pixel.
11. The method according to any one of claims 1 to 10, wherein,

    the original image includes a live image captured by an augmented reality AR device, and

    The AR device displays the target image.
12. The method according to any one of claims 1 to 10, wherein the original image comprises:

    an image of a target person, wherein the target object includes at least one of a human hair area, a human skin area, and at least part of a clothing area in the target person image; or,

    A target object image, wherein the target object is at least a part of the object area in the target object image.
13. An image processing device, comprising:

    an acquisition module, configured to acquire a probability value that at least part of the pixels in the original image are target pixels, wherein the target pixels are pixels belonging to the target object in the original image;

    A determination module, configured to determine the fusion color value corresponding to the pixel point based on the probability value that the pixel point is the target pixel point and the pre-selected target color value for any pixel point in the at least part of the pixel point ;

    A generating module, configured to adjust the color values of the at least part of the pixel points in the original image by using the fusion color values corresponding to the at least part of the pixel points respectively, to obtain a target image.
14. An electronic device, comprising:

    a memory storing machine-readable instructions,

    a processor for executing machine-readable instructions stored in the memory,

    Wherein, when the machine-readable instructions are executed by the processor, the processor executes the steps in the image processing method according to any one of claims 1 to 12.
15. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is run by a computer device, the computer device executes the image processing method according to any one of claims 1 to 12. step.

Images