CN114581979A - Image processing method and device - Google Patents

Image processing method and device Download PDF

Info

Publication number
CN114581979A
CN114581979A CN202210198969.9A CN202210198969A CN114581979A CN 114581979 A CN114581979 A CN 114581979A CN 202210198969 A CN202210198969 A CN 202210198969A CN 114581979 A CN114581979 A CN 114581979A
Authority
CN
China
Prior art keywords
face
image
processing
determining
point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210198969.9A
Other languages
Chinese (zh)
Inventor
王萍萍
李彬
张磊
王海新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202210198969.9A priority Critical patent/CN114581979A/en
Publication of CN114581979A publication Critical patent/CN114581979A/en
Priority to PCT/CN2023/077278 priority patent/WO2023165369A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/77Retouching; Inpainting; Scratch removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an image processing method and device, and relates to the technical field of computers. One embodiment of the method comprises: acquiring an image to be processed; determining a face central point and a face area in the image to be processed; determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to a target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point; and performing face beautifying processing on the face area by using the processing strength of each target pixel point. This embodiment enables a good transition between processed and unprocessed regions in the processed image.

Description

Image processing method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an image processing method and apparatus.
Background
Face beautification processing techniques have been commonly applied in photography, video communication, live broadcasting, and the like. The face beautifying treatment can be carried out in a face masking mode, namely, the mask for removing the five sense organs of the face is adopted, and the beautifying treatment is carried out on the parts of the face except the five sense organs. The face beautifying treatment can eliminate spots, blemishes, mottles, and the like on the skin portion of the face. However, the boundary of the image processed by the face mask has a very obvious boundary sense, so that the effect of beautifying the processed image is influenced.
Disclosure of Invention
In view of this, embodiments of the present invention provide an image processing method and apparatus, which can make a processed region and an unprocessed region in a processed image have good transition.
In a first aspect, an embodiment of the present invention provides an image processing method, including:
acquiring an image to be processed;
determining a face central point and a face area in the image to be processed;
determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face central point;
and performing face beautifying processing on the face area by using the processing strength of each target pixel point.
Optionally, the determining a center point of a face and a face region in the image to be processed includes:
determining a first key point of the face region, and determining the first key point as the face central point;
determining a second key point and a third key point of the face region, and determining a key distance between the second key point and the third key point;
determining the face diameter by using the key distance and a preset coefficient;
and determining the face area of the image to be processed by taking the first key point as a circle center and the face diameter as a diameter.
Optionally, the performing, by using the processing intensity of each target pixel point, face beautification processing on the face region includes:
determining a current area from the face area;
determining whether the current region is a skin color region;
and under the condition that the current area is a skin color area, performing face beautifying processing on the current area by using the processing intensity of each pixel point in the current area.
Optionally, the determining the center point of the face and the face region corresponding to the image to be processed includes:
performing down-sampling processing on the image to be processed;
and determining the center point of the face and the face area in the image after the down-sampling processing.
Optionally, after performing face beautification on the face region by using the processing strength of each target pixel point, the method further includes:
and performing fusion processing on the image subjected to face beautification processing and the image to be processed to generate a face beautifying image.
Optionally, before performing the down-sampling processing on the image to be processed, the method further includes:
acquiring terminal parameters of a target terminal;
and executing the step of performing down-sampling processing on the image to be processed under the condition that the terminal parameter meets a preset condition.
Optionally, after performing face beautification on the face region by using the processing strength of each target pixel point, the method further includes:
determining the skin color brightness of each face subarea in the image after the face beautifying processing;
determining the adjustment coefficient of each face subarea according to the skin color brightness of each face subarea;
and whitening the corresponding face subareas by adopting the adjustment coefficients of the face subareas.
In a second aspect, an embodiment of the present invention provides an image processing apparatus, including:
the image acquisition module is used for acquiring an image to be processed;
the region determining module is used for determining a face central point and a face region in the image to be processed;
the intensity determination module is used for determining the processing intensity of each target pixel point in the face region, the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point;
and the processing module is used for performing face beautifying processing on the face area by using the processing strength of each target pixel point.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any of the embodiments described above.
In a fourth aspect, an embodiment of the present invention provides a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method of any one of the above embodiments.
One embodiment of the above invention has the following advantages or benefits: the processing intensity of each target pixel point in the face area is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point. In the face region, the processing intensity is gradually reduced from the center point of the face to the outside, so that the processing intensity of the pixel points at the boundary of the processing region is smaller. Therefore, the treated region and the untreated region can be well transited.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of an exemplary application scenario in which embodiments of the present invention may be applied;
FIG. 2 is a flow chart of an image processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating another image processing method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of face keypoints for use in a beautification processing area according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating another exemplary image processing method according to an embodiment of the present invention;
FIG. 6 is a diagram of a Render-To-Texture process flow at one RTT provided by one embodiment of the present invention;
FIG. 7 is a flowchart illustrating a further image processing method according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
FIG. 1 is a schematic diagram of an exemplary application scenario in which embodiments of the present invention may be applied. As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, and 103 may be deployed with an image display client, a photographing client, a live broadcast client, or a browser. The terminal devices 101, 102, 103 may interact with the server 105 using a client or browser. The terminal devices 101, 102, 103 may be cell phones, notebooks, tablets, laptop portable computers, etc.
The user issues a request to view live or images to the terminal apparatuses 101, 102, 103. The terminal devices 101, 102, 103 send service requests to the server 105 via the network 104. After receiving the service request, the server 105 acquires an image to be processed; determining a face central point and a face area in the image to be processed; determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to a target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point; and performing face beautifying processing on the face area by using the processing strength of each target pixel point, and returning the face beautifying processed image to the terminal equipment 101, 102 and 103.
It should be noted that the image processing method provided by the embodiment of the present invention is generally executed by the server 105, and accordingly, the image processing apparatus is generally disposed in the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present invention, and as shown in fig. 2, the method includes:
step 201: and acquiring an image to be processed.
The image to be processed contains a face region. The image to be processed may include one face region or a plurality of face regions. The image to be processed may be a stored picture, a picture previewed during photographing, or a video frame captured from a video, etc.
Step 202: and determining a face central point and a face area in the image to be processed.
The center point of the face is the center position of the face area. The center point of the face can be set according to specific needs. For example, the nose tip position may be set as the face center point, and the middle point between the eyebrow center and the labial bead may be set as the face center point. The face detection can be carried out on the image to be processed through a face recognition technology to obtain a face area, and then the central point of the face area is calculated.
The face region is a region in which a face is displayed in the image to be processed. The image to be processed can be identified to obtain the face region. The image to be processed can also be input into the face key point model, a plurality of key points of the face are obtained, and then the face area is determined according to the plurality of key points. The face key point model can be a Blazeface model, an MT3DMM model and the like.
Step 203: determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point.
The face beautifying processing is an image processing method, can eliminate flaws such as spots or variegates in a face image, and can enable the face of a person to be finer and smoother. In the face image, spots, acne marks, birthmarks, scars, and the like on the face of a human face are generally displayed as high-frequency information. The face beautifying treatment can filter out high-frequency information such as acne marks, birthmarks, scars and the like. And then acquiring low-frequency information corresponding to the smooth skin, and filling the area of the high-frequency information by using the low-frequency information. Because the facial beautification treatment can not be carried out on the facial features, the mask for removing the facial features is usually adopted in the prior art, and only the beautification treatment is carried out on the facial skin area. However, this again makes the boundary between the treated and untreated areas at the mask boundary apparent.
The processing intensity reflects the degree of flaw removal and the fineness of the human face. The processing intensity can be determined by the frequency intensity of high-frequency information in the face image to be filtered. Generally, the lower the intensity of the frequency to be filtered, the stronger the treatment intensity, the fewer the blemishes, and the more delicate the skin.
In an embodiment of the present invention, the face beautification processing effect can be realized by a bilateral filtering algorithm, and beautification processing is performed on a face area, which requires to realize beautification processing of a full range of a face. The boundary between the human face and the environment has gradually reduced processing intensity to prevent abrupt change. The reduction of the processing intensity can be controlled by an inverse proportion function, and the closer to the center point of the face, the greater the processing intensity, and within a certain range (related to the size and the posture of each face, etc.), the farther from the center point of the face, the smaller the processing intensity. Thereby realizing the gradual reduction of the processing intensity from the center to the outside and presenting the transition effect.
Step 204: and performing face beautifying treatment on the face area by using the processing strength of each target pixel point.
In the embodiment of the invention, the processing intensity of each target pixel point in the face region is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point. From the center point of the face outwards, the processing intensity gradually decreases. Specifically, the processing intensity is gradually decreased by taking the center point of the face as a center, the processing intensity of the center point of the face is the maximum, the processing intensity of the pixel point at the boundary between the processed area and the unprocessed area for beautifying the face is equal to or less than a preset intensity value, and the preset intensity value may be zero or a value close to zero. So that the processed area and the unprocessed area in the finally beautified processed image are in good transition.
Development of the mobile terminal live broadcast basic beautifying function is usually realized based on opengles. In the case of a Personal Computer (PC) live broadcast beauty scheme, PC live broadcast is based on an obs open source PC streaming tool, data docking is considered when developing a beauty function, and an upper layer live broadcast streaming tool is based on a multimedia programming interface directx (direct extension) created by windows microsoft corporation.
DirectX is a multimedia programming interface created by microsoft corporation and is an application program interface. DirectX can enable a game or a multimedia program taking windows as a platform to obtain higher execution efficiency, strengthen 3D graphics and sound effects, provide a common hardware driving standard for designers, enable game developers not to write different drivers for each brand of hardware, and reduce the complexity of hardware installation and setting for users.
Problems may arise due to interfacing opengl with Directx. Therefore, the mobile terminal cannot be directly applied to the PC terminal based on the functions of opengl ES, such as skin care. In order to avoid too many subsequent problems, the embodiment of the invention provides a method for realizing PC live broadcast beauty optimization by using DirectX, and a bottom layer beauty method or an image processing method is developed by using DirectX. The PC live broadcast is based on an obs open source PC stream pushing tool, and data butt joint is considered when the beautifying function is developed. The upper layer is developed by using DirectX to realize better compatibility, and the provided beauty sdk is also developed by using DirectX.
In the beautifying function, the face beautifying processing of the static picture carries out the area beautifying processing according to the result of the skin color detection algorithm, and a better effect can be achieved. In the face beautifying process, a face beautifying processing area is based on a skin color detection result, each frame of a picture is changed in a live broadcast process, the skin color detection effect is poor, the processing area is too large, and the whole live broadcast picture is too fuzzy.
Based on the method, the human face area in the image to be processed can be accurately determined, and the situation that a live broadcast picture is not clear is prevented. Fig. 3 is a schematic flowchart of another image processing method according to an embodiment of the present invention, and as shown in fig. 3, the method includes:
step 301: and acquiring an image to be processed.
Step 302: and determining a first key point of the face area, and determining the first key point as a face central point.
And determining a target area in the image to be processed by utilizing an image recognition technology. The target area is an area that may include a human face. The target area is input into the face key point model, and the coordinate information, the face pose information and the like of a plurality of key points of the face can be obtained. The first key point is used for representing the central point of the face area. The first key point can be set according to specific requirements. For example, for a face model with 68 face keypoints, the face keypoint with the position number of 34 may be selected as the first keypoint. For the face model with 49 face key points, the face key point with the position serial number of 13 can be selected as the first key point.
Step 303: and determining a second key point and a third key point of the face area.
The second key point and the third key point are respectively positioned at two key points which are symmetrical to the center point of the face from two sides. For example, the second key point and the third key point may be two key points in the face region that are farthest away from the center point of the face. The second key point and the third key point can be set according to specific requirements. For example, for a face model with 68 face key points, the face key points with position numbers 1 and 17 may be selected as the second key point and the third key point, respectively. For the face model with 49 personal face key points, the face key points with the position numbers of 1 and 10 can be respectively selected as the second key point and the third key point.
Step 304: and determining the face area according to the first key point, the second key point and the third key point.
The first key point is used for representing the central point of the face area. The second key point and the third key point are respectively positioned at two key points which are symmetrical to the center point of the face from two sides. And determining the face area by using the first key point, the second key point and the third key point.
Step 305: determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point.
Step 306: and performing face beautifying treatment on the face area by using the processing strength of each target pixel point.
In the prior art, people in the live broadcast process are not still pictures, and the face beautifying processing of a result area is not accurate only by using a skin color detection algorithm, and anchor clothing, a live broadcast room environment, displayed commodities and the like can be identified as skins, so that the result is that the live broadcast picture processing area is too large, the environment and the displayed commodities are processed, the shopping experience of a user is influenced, and the conversion rate of the live broadcast room is not favorably improved. According to the scheme of the embodiment of the invention, the first key point, the second key point and the third key point are selected from the plurality of face key points, and the beautification processing area is limited by using the first key point, the second key point and the third key point, so that the problem that the commodity is mistakenly identified to cause blurring is avoided.
The first method is to adopt a face mask mode, and adopt a mask for removing five sense organs of the face by one piece, beautify the part of the face except the five sense organs, and then mix the beautified part with an original image, so that the effect of only processing the face can be obtained, but the defect is that very obvious boundary lines (one side is processed and the other side is unprocessed, so that the boundary sense is very obvious) exist at the boundary of the mask. The second is that the whole image adopts a skin color detection algorithm, and only the area which passes through skin color detection is beautified, so that the method is easy to cause the environment to be mistakenly identified as skin, and further causes the part of the environment except the skin to be beautified, thereby causing the environment or the commodity to be blurred.
Based on this, the embodiment of the present invention designs a method for determining a face region: determining a key distance between the second key point and the third key point; determining the face diameter by using the key distance and a preset coefficient; and determining the face area of the image to be processed by taking the first key point as a circle center and the face diameter as a diameter.
The method only beautifies a circular area which takes the center of the face as the center of a circle and is within a certain radius, and can maintain the original definition of the environment and the commodity while realizing good beautifying effect. The specific scheme is as follows: the key points of the face center are obtained, the strength is uniformly reduced from the center to the outside, the processed area and the unprocessed area are in good transition, the facial regions are removed by using skin color detection in the area, and the smooth operation is refused if the facial regions are not skin color areas, so that the face beautifying processing of the face area is realized, the blurring of organs such as eyebrows, eyes and the like is avoided, and the face beautifying effect is supported. In addition, the beautified image can be fused with the original image, so that the beautification effect of the face is ensured, and meanwhile, the commodity information is prevented from being fuzzy.
Fig. 4 is a schematic diagram of face key points used in a beautification processing area according to an embodiment of the present invention. As shown in fig. 4, after the face key point information is obtained, the face key point with the face center position serial number of 46 is taken as a first key point, which is the center point of the drawing circle. The 2, 30 key points on both sides of the cheek are taken as the second key point and the third key point respectively. And taking the result of multiplying the distance between the second key point and the third key point by a preset coefficient as the diameter, and drawing the circular area. Only in the circular area, the processing intensity from the center point of the face outwards is uniformly reduced, so that the processed area and the unprocessed area show good transition effect.
In one embodiment of the present invention, the face region may also be determined by: determining face pose information of the face area; and determining the face area by using the first key point, the second key point, the third key point and the face pose information. Because the human face can turn to action such as in the live broadcast process, in order to guarantee the accuracy of result, the pose information of the human face needs to be added into the calculation, and therefore a more accurate human face area is obtained. For example, using image recognition techniques, a face region is determined in the image to be processed. And inputting the face region into a face key point model, and acquiring coordinate information, face pose information and the like of a plurality of key points of the face. And determining a face central point from the plurality of key points according to the face pose information.
In addition, when the PC side is in live broadcasting, the resolution ratio of the camera can be switched to be multiple, and in order to obtain an elliptical area consistent with the direction of the face, the aspect ratio of the resolution ratio can be included in the calculation of the face area.
From the perspective of the algorithm, the face beautification processing is mainly realized by filtering, a skin color detection algorithm and the like. The face beautifying treatment is to filter out high frequency information such as acne marks, birthmarks, scars and the like by using a filter and fill the high frequency information with low frequency information such as smooth skin and the like. After the key points of the face are used for delineating the face area, the face mask can be obtained by combining a skin detection algorithm, and then the face mask is mixed with a filtering result, so that the beautifying effect is realized, the original image is used in other areas such as the background, and the details can be reserved. The live broadcast tape is better focused on the beautifying effect of the face, and in terms of balance, the method can not beautify all exposed skins, but better ensures the requirements of beautifying the face and keeping the details of goods. Algorithms such as skin color detection and the like are only carried out in the area, so that the environmental blur is reduced, the calculation pressure is reduced, the final face beautifying effect is enhanced, and the algorithm performance is improved.
The beauty algorithm realized by the embodiment of the invention can be summarized into low-resolution filtering and skin color detection and high-resolution high-contrast and whitening. The high contrast retaining algorithm is to retain a place with a large contrast in the original image, for example, in a human face image, the place with the large contrast is five sense organs, the original image is usually smoothed when the face is beautified, but the detail information of the image is lost after the processing, so that the detail information of the image can be retained through the high contrast retaining or other high-pass filters, and then the smoothed image and the high-frequency image are subjected to light mixing, so that a better effect can be obtained.
In an embodiment of the present invention, the performing facial beautification processing on the face area by using the processing strength of each target pixel point includes: determining a current area from the face area; determining whether the current region is a skin color region; and under the condition that the current area is a skin color area, performing face beautifying processing on the current area by using the processing intensity of each pixel point in the current area.
The filtering algorithm adopted by the image processing method of the embodiment of the invention is a bilateral filtering algorithm. The main process is as follows: taking out the green channel, and carrying out Gaussian blurring on the green channel to obtain a blurred value sampleColor; the original green channel value minus sampleColor, plus 0.5 (i.e., 128), both steps, i.e., high contrast retention in PS (mentioned above); the result is subjected to highlight treatment for 3-5 times, so that the noise is more prominent; the gray scale value of the original image is calculated by the formula 0.299R + 0.587G + 0.114B, and after the gray scale image is obtained, the gray scale value is used as a threshold value to exclude non-skin portions, and the original image is synthesized with the result image according to the gray scale value calculation.
Even if the color of the skin area changes greatly, the skin color is concentrated in a relatively narrow area, and the skin color detection is performed by using the principle. The skin color detection method in the embodiment of the invention adopts a mixed Gaussian probability model, applies the skin color model of a YCbCr color space, utilizes the characteristics of mixed Gaussian distribution to compare and segment skin colors, and adopts an image skin region segmentation algorithm based on the Gaussian skin color model.
Firstly, selecting a color space model: the Gaussian mixture model was chosen for the following reasons: skin color histograms with different ethnicities do not fully satisfy unimodal gaussian distributions, which can be accurately represented by studying multimodal gaussian distributions, and the following mixed gaussian model is proposed:
Figure BDA0003526879580000111
in the above formula, p (x) is the mixed probability density of skin color pixels in the color space, p (x | w)i) Is the probability density of a component, p (w)i) I is the prior probability of the component, 1, 2, … …, and m is the number of components of the mixture density. The model indicates that each pixel density of skin color belongs to a mixture of probability densities. The main difficulty in this model is the parameter estimation of the gaussian mixture model, and the parameter estimation often adopts the maximum likelihood-based algorithm. The algorithm needs iteration, and the convergence rate of the iteration is influenced by the initial value of the parameter and is divided intoThe quantity density number relation is large, and the more the components are, the more the operation is complex. The method has high detection efficiency of the skin color and low false detection rate, but the determination of the model (namely the parameter estimation of the model) is difficult, the speed is relatively slow, and the method is not suitable for rapid skin color detection.
Establishing a skin color model based on a Gauss skin color model of a YCbCr space:
Figure BDA0003526879580000122
wherein, x is (CbCr)T
By calculation, the values of m and C are obtained as follows:
m=[117.4316 148.5599]
Figure BDA0003526879580000121
fig. 5 is a schematic flowchart of another image processing method according to an embodiment of the present invention, as shown in fig. 5, the method includes:
step 501: and acquiring an image to be processed.
Step 502: and performing down-sampling processing on the image to be processed.
The down-sampling process is used to reduce the resolution of the image to be processed. The image to be processed may be down-sampled in a variety of ways. If the image to be processed can be directly subjected to down-sampling processing to generate an image conforming to the preset resolution, the down-sampling processing can also be realized by utilizing the RTT technology.
Fig. 6 is a schematic diagram of a processing flow at an RTT according to an embodiment of the present invention. The RTT technique actually uses a 2D texture as a rendering target, but generally requires a newly created 2D texture. At the same time, this texture can also be bound to the Shader Resource View (SRV) for use by the shader, i.e., the texture that was originally used as output is now used as input.
In order to reduce the amount of calculation, RTT technique is used after taking image data of a live view. With the RTT technique of DirectX, textures meeting preset requirements are set as rendering targets, for example, a 2D texture of 1/2 with width and height of each image to be processed is used. And combining the obtained target area, calculating on the small graph, reducing the total calculation amount and improving the performance of a gpu (graphics processing unit).
Step 503: and determining a face central point and a face area in the image subjected to the down-sampling processing.
Step 504: and determining a face central point and a face area in the image to be processed.
Step 505: determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point.
Step 506: and performing face beautifying treatment on the face area by using the processing strength of each target pixel point.
Step 507: and performing fusion processing on the image subjected to face beautifying processing and the image to be processed to generate a beautifying image.
The face beautifying algorithm consumes gpu performance, and particularly in the general model of a video card, in order to improve the performance, after a face area is obtained, down-sampling processing is performed on an image to be processed, and a complex face beautifying algorithm is operated on the down-sampled image.
The image after down-sampling is fuzzy, and in order to solve the problem, the image result after face down-sampling processing is mixed with the image to be processed so as to ensure the definition of the region except the face. Specifically, the image after face beautification processing and the image to be processed may be fused by adopting a linear mixing method, and the formula is as follows:
and the pixel value of the beautified image is equal to the pixel value of the image to be processed (1-processing intensity value) + the pixel value of the image after the face beautification processing is equal to the processing intensity value.
In an embodiment of the present invention, before performing the down-sampling processing on the image to be processed, the method further includes: acquiring terminal parameters of a target terminal; and executing the step of performing down-sampling processing on the image to be processed under the condition that the terminal parameter meets a preset condition. When the terminal parameters meet the preset conditions, in order to maintain good performance thereof, a down-sampling process is used. When the terminal parameter does not meet the preset condition, for example, the resolution selected by the user is low, and the performance of the terminal is high, the down-sampling processing is not needed.
When the algorithm is implemented, hardware information of gpu, cpu and the like of a user and a video resolution currently selected by the user are captured, when the user selects a lower resolution (for example, 640 × 480), downsampling is not performed (at the moment, a video output image is fuzzy, and a machine meets performance requirements), and downsampling of the beauty algorithm is started only when the model of the user is lower than research data and a higher resolution is selected, so that the performance of selecting a high-resolution image is improved, and meanwhile, a low-resolution image is prevented from being more fuzzy.
In an embodiment of the present invention, after performing face beautification on the face region by using the processing strength of each target pixel point, the method further includes: determining the skin color brightness of each face subarea in the image after the face beautifying processing; determining an adjusting coefficient of each face subarea according to the skin color brightness of each face subarea; and whitening the corresponding face subareas by adopting the adjustment coefficients of the face subareas.
The whitening treatment is used to make the skin of the face in the image fine and fair. The skin whitening treatment can be performed by using an LUT method, which is a method of adjusting brightness/contrast, a curve, color balance, and the like in software such as PS, or a method of adjusting a certain brightness curve to generate a corresponding LUT, and using an LUT filter method to achieve skin whitening. Its advantages are use of colour filter LUT, high speed and real-time processing.
The whole face area can be divided into a plurality of face subareas. The human face sub-region is a region in the human face image having the same or similar image attributes. The face recognition can be carried out on the image to be processed by adopting an artificial intelligence recognition technology so as to obtain a plurality of face subareas in the image to be processed.
The skin tone brightness of each face sub-region can be determined in a variety of ways. For example, the brightness values of all pixel points or all key points in a certain face sub-region are obtained, and the average value or weighted sum of the brightness values can be used as the skin color brightness of the face sub-region. The skin tone brightness formula is shown below:
Figure BDA0003526879580000141
where w (x, y) is the skin color brightness of the original pixel (x, y), v (x, y) is the skin color brightness after enhancement, β is the adjustment coefficient, and the larger the value, the stronger the whitening intensity.
And determining an adjusting coefficient corresponding to the skin color brightness according to the preset corresponding relation. Generally, the skin color brightness is inversely related to the adjustment coefficient, i.e., the smaller the skin color brightness, the larger the adjustment coefficient, and the larger the whitening intensity. The larger the skin color brightness is, the smaller the adjustment coefficient is and the smaller the whitening intensity is.
And further whitening the image subjected to the face beautifying treatment, so that the finally generated beautifying image has a better display effect.
To facilitate an understanding of the method of the present example, a specific cosmetic embodiment of the present invention is set forth below. The image processing method in the embodiment of the invention mainly refers to face beautifying and whitening. The face beautifying treatment needs to treat the face skin area to be fine and smooth. Whitening requires a fair and ruddy treatment of the skin area. Fig. 7 is a flowchart illustrating a further image processing method according to an embodiment of the present invention. As shown in fig. 7, a flow chart for implementing a beauty effect using DirectX. The method comprises the steps of inputting camera images and face key point data, performing down-sampling image processing and skin color mapping whitening on main rendering processes, and outputting a result which is an image after the face is beautified. After camera data are taken, textures are drawn on 1/4 small graphs through an RTT technology, then face beautifying algorithms are carried out on the small graphs, and the images subjected to face beautifying processing including filtering, skin color detection and the like are subjected to overall fuzzy processing, so that the whole images are not completely bright, and the images need to be sharpened. And then the obtained result image is fused with the skin-color mapped whitening result to obtain a beautifying output image with superposed beautifying and whitening effects on the face.
In the embodiment of the invention, the PC machines have multiple models and large performance difference, in order to enable the PC beautifying function to be supported by as many models as possible, the performance of a beautifying algorithm needs to be improved, an RTT (round trip time) technology is used in the PC beautifying algorithm, the technology is a down-sampling scheme which can be used for realizing beautifying, and algorithms with relatively high consumption performance such as high contrast and the like are realized on a small graph so as to improve the performance of a PC direct broadcast beautifying function module. The technical principle is that 2D textures with the width and the height of the original image 1/2 are used as rendering targets, calculation is carried out on small images, the total calculation amount is reduced, and the gpu performance is improved.
The face beautification treatment usually adopts a scheme of face masking, but the scheme can cause the edges of treated and non-treated areas to be obvious, so the effect is not ideal. The skin color detection algorithm can avoid the transition of the hardness of the edge of the face, has good effect in detecting a static picture, but has poor picture stability in the live broadcast process, and a large number of areas similar to skin colors exist in commodities and the like, so that the detection result is not accurate enough, and the result is that objects in some environments are beautified and processed, and the environmental area is fuzzy. In order to improve the situation, the scheme of the embodiment of the invention adopts a method that the face beautifying treatment is only carried out on a circular area which takes the center of the face as the center of a circle and is within a certain radius, so that the original definition of the environment and the commodity can be maintained while the good face beautifying effect is realized. The specific scheme is as follows: the key points of the face center are obtained, the strength is uniformly reduced from the center to the outside, the processed and unprocessed areas are in good transition, the facial regions are removed by using skin color detection in the areas, and the smooth processing is refused if the processed and unprocessed areas are not skin color areas, so that the face beautifying processing of the face area is realized, the blurring of organs such as eyebrows, eyes and the like is avoided, and the face beautifying effect is supported. And fusing the beautified image with the original image, thereby ensuring that the face beautification effect is obtained and avoiding the blurring of commodity information.
In addition, the scheme of the embodiment of the invention also obtains the data of the cpu character number, the gpu model number, the resolution selected by the camera and the like in the terminal, and is used for assisting in improving the PC beautifying performance and simultaneously maintaining good image definition. And judging by utilizing a white list and other forms, when the cpu and the gpu meet a certain threshold of a corresponding skyhook, using a down-sampling resolution for maintaining good performance of the cpu and the gpu, and when the resolution selected by a user is low, judging that the user machine can maintain good performance and does not need to use the user machine.
Fig. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention, and as shown in fig. 8, the apparatus includes:
an image obtaining module 801, configured to obtain an image to be processed;
a region determining module 802, configured to determine a center point of a face and a face region in the image to be processed;
an intensity determining module 803, configured to determine processing intensity of each target pixel in the face region, where the processing intensity of the target pixel is inversely proportional to a target distance corresponding to the target pixel, and the target distance is a distance between the target pixel and the face center point;
and the processing module 804 is configured to perform face beautification processing on the face area by using the processing strength of each target pixel point.
Optionally, the area determining module 802 is specifically configured to:
determining a first key point of the face region, and determining the first key point as the face central point;
determining a second key point and a third key point of the face region, and determining a key distance between the second key point and the third key point;
determining the face diameter by using the key distance and a preset coefficient;
and determining the face area of the image to be processed by taking the first key point as a circle center and the face diameter as a diameter.
Optionally, the processing module 804 is specifically configured to:
determining a current area from the face area;
determining whether the current region is a skin color region;
and under the condition that the current area is a skin color area, performing face beautifying processing on the current area by using the processing intensity of each pixel point in the current area.
Optionally, the area determining module 802 is specifically configured to:
performing down-sampling processing on the image to be processed;
and determining the center point of the face and the face area in the image subjected to the down-sampling processing.
Optionally, the apparatus further comprises:
and a fusion module 805, configured to perform fusion processing on the image after the face beautification processing and the image to be processed to generate a beauty image.
Optionally, the apparatus further comprises:
a parameter obtaining module 806, configured to obtain a terminal parameter of the target terminal.
Optionally, the apparatus further comprises:
the beautifying module 806 is configured to perform whitening processing on the image after the face beautifying processing, and generate a beautifying image corresponding to the image to be processed.
An embodiment of the present invention provides an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method of any of the embodiments described above.
Referring now to FIG. 9, shown is a block diagram of a computer system 900 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 9, the computer system 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the system 900 are also stored. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output portion 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. A drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911. The above-described functions defined in the system of the present invention are executed when the computer program is executed by a Central Processing Unit (CPU) 901.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: the device comprises an image acquisition module, an area determination module, an intensity determination module and a processing module. The names of these modules do not in some cases constitute a limitation to the module itself, and for example, the image acquisition module may also be described as a "module that acquires an image to be processed".
As another aspect, the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
acquiring an image to be processed;
determining a face central point and a face area in the image to be processed;
determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to a target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point;
and performing face beautifying processing on the face area by using the processing strength of each target pixel point.
According to the technical scheme of the embodiment of the invention, the processing intensity of each target pixel point in the face region is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point. From the center point of the face outwards, the processing intensity is gradually reduced, so that the processed area and the unprocessed area can be in good transition.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. An image processing method, comprising:
acquiring an image to be processed;
determining a face central point and a face area in the image to be processed;
determining the processing intensity of each target pixel point in the face region, wherein the processing intensity of the target pixel point is inversely proportional to a target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point;
and performing face beautifying processing on the face area by using the processing strength of each target pixel point.
2. The method according to claim 1, wherein the determining the center point of the face and the face region in the image to be processed comprises:
determining a first key point of the face region, and determining the first key point as the face central point;
determining a second key point and a third key point of the face region, and determining a key distance between the second key point and the third key point;
determining the face diameter by using the key distance and a preset coefficient;
and determining the face area of the image to be processed by taking the first key point as a circle center and the face diameter as a diameter.
3. The method according to claim 1, wherein the performing facial beautification processing on the face region by using the processing intensity of each target pixel point comprises:
determining a current area from the face area;
determining whether the current region is a skin color region;
and under the condition that the current area is a skin color area, performing face beautifying processing on the current area by using the processing intensity of each pixel point in the current area.
4. The method according to claim 1, wherein the determining the center point of the face and the face region corresponding to the image to be processed comprises:
performing down-sampling processing on the image to be processed;
and determining the center point of the face and the face area in the image subjected to the down-sampling processing.
5. The method according to claim 4, wherein after performing face beautification on the face region by using the processing intensity of each target pixel point, the method further comprises:
and performing fusion processing on the image subjected to face beautification processing and the image to be processed to generate a face beautifying image.
6. The method according to claim 4, wherein before the down-sampling the image to be processed, the method further comprises:
acquiring terminal parameters of a target terminal;
and executing the step of performing down-sampling processing on the image to be processed under the condition that the terminal parameter meets a preset condition.
7. The method of claim 1, wherein after performing a face beautification process on the face region by using the processing intensity of each of the target pixels, the method further comprises:
determining the skin color brightness of each face subarea in the image after the face beautifying processing;
determining an adjusting coefficient of each face subarea according to the skin color brightness of each face subarea;
and whitening the corresponding face subareas by adopting the adjustment coefficients of the face subareas.
8. An image processing apparatus characterized by comprising:
the image acquisition module is used for acquiring an image to be processed;
the region determining module is used for determining a face central point and a face region in the image to be processed;
the intensity determination module is used for determining the processing intensity of each target pixel point in the face region, the processing intensity of the target pixel point is inversely proportional to the target distance corresponding to the target pixel point, and the target distance is the distance between the target pixel point and the face center point;
and the processing module is used for performing face beautifying processing on the face area by using the processing strength of each target pixel point.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN202210198969.9A 2022-03-01 2022-03-01 Image processing method and device Pending CN114581979A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210198969.9A CN114581979A (en) 2022-03-01 2022-03-01 Image processing method and device
PCT/CN2023/077278 WO2023165369A1 (en) 2022-03-01 2023-02-21 Image processing method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210198969.9A CN114581979A (en) 2022-03-01 2022-03-01 Image processing method and device

Publications (1)

Publication Number Publication Date
CN114581979A true CN114581979A (en) 2022-06-03

Family

ID=81776416

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210198969.9A Pending CN114581979A (en) 2022-03-01 2022-03-01 Image processing method and device

Country Status (2)

Country Link
CN (1) CN114581979A (en)
WO (1) WO2023165369A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023165369A1 (en) * 2022-03-01 2023-09-07 北京沃东天骏信息技术有限公司 Image processing method and apparatus
CN116993133A (en) * 2023-09-27 2023-11-03 尚云(广州)信息科技有限公司 Intelligent work order system based on face recognition

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104899575A (en) * 2015-06-19 2015-09-09 南京大学 Human body assembly dividing method based on face detection and key point positioning
CN108428214B (en) * 2017-02-13 2022-03-08 斑马智行网络(香港)有限公司 Image processing method and device
CN107730465B (en) * 2017-10-09 2020-09-04 武汉斗鱼网络科技有限公司 Face beautifying method and device in image
JP6590047B2 (en) * 2018-08-08 2019-10-16 カシオ計算機株式会社 Image processing apparatus, imaging apparatus, image processing method, and program
CN111145086A (en) * 2019-12-27 2020-05-12 北京奇艺世纪科技有限公司 Image processing method and device and electronic equipment
CN112784773B (en) * 2021-01-27 2022-09-27 展讯通信(上海)有限公司 Image processing method and device, storage medium and terminal
CN114581979A (en) * 2022-03-01 2022-06-03 北京沃东天骏信息技术有限公司 Image processing method and device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023165369A1 (en) * 2022-03-01 2023-09-07 北京沃东天骏信息技术有限公司 Image processing method and apparatus
CN116993133A (en) * 2023-09-27 2023-11-03 尚云(广州)信息科技有限公司 Intelligent work order system based on face recognition
CN116993133B (en) * 2023-09-27 2024-01-26 尚云(广州)信息科技有限公司 Intelligent work order system based on face recognition

Also Published As

Publication number Publication date
WO2023165369A1 (en) 2023-09-07

Similar Documents

Publication Publication Date Title
US11132824B2 (en) Face image processing method and apparatus, and electronic device
CN108229278B (en) Face image processing method and device and electronic equipment
JP4461789B2 (en) Image processing device
CN106846241B (en) Image fusion method, device and equipment
CN113327278B (en) Three-dimensional face reconstruction method, device, equipment and storage medium
WO2023165369A1 (en) Image processing method and apparatus
WO2020177434A1 (en) Image processing method and apparatus, image device, and storage medium
CN112258440B (en) Image processing method, device, electronic equipment and storage medium
Kim et al. Low-light image enhancement based on maximal diffusion values
WO2014076744A1 (en) Image processing device and image processing method
CN113240783B (en) Stylized rendering method and device, readable storage medium and electronic equipment
CN115100334B (en) Image edge tracing and image animation method, device and storage medium
CN116363288A (en) Rendering method and device of target object, storage medium and computer equipment
CN113313631B (en) Image rendering method and device
WO2024125328A1 (en) Live-streaming image frame processing method and apparatus, and device, readable storage medium and product
CN114049290A (en) Image processing method, device, equipment and storage medium
CN114862729A (en) Image processing method, image processing device, computer equipment and storage medium
CN113344837A (en) Face image processing method and device, computer readable storage medium and terminal
Mould et al. Stylized black and white images from photographs
US20220398704A1 (en) Intelligent Portrait Photography Enhancement System
CN113379623B (en) Image processing method, device, electronic equipment and storage medium
CN115311403A (en) Deep learning network training method, virtual image generation method and device
CN113223128B (en) Method and apparatus for generating image
CN114187202A (en) Image processing method, apparatus, device, storage medium, and program product
US9563940B2 (en) Smart image enhancements

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination