WO2023142645A1

WO2023142645A1 - Image processing method and apparatus, and electronic device, storage medium and computer program product

Info

Publication number: WO2023142645A1
Application number: PCT/CN2022/134617
Authority: WO
Inventors: 苏柳
Original assignee: 上海商汤智能科技有限公司
Priority date: 2022-01-28
Filing date: 2022-11-28
Publication date: 2023-08-03
Also published as: CN114463212A

Abstract

The present disclosure relates to an image processing method and apparatus, and an electronic device, a storage medium and a computer program product. The method comprises: determining a first mask image and a second mask image of a facial image, wherein the first mask image and the second mask image are used for representing a first part area and a second part area, respectively, in the facial image, and the second part area is in the first part area; acquiring lip processing information comprising area information and beautification parameter information; adjusting the range of the second part area in the second mask image according to the area information and the first mask image; and beautifying the facial image according to the beautification parameter information and the adjusted second mask image, so as to obtain a target image in which the color of the second part area is adjusted.

Description

Image processing method, device, electronic device, storage medium and computer program product

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application number 202210106872.0 submitted on January 28, 2022, the applicant is Beijing Datian Mian Baitang Technology Co., Ltd., and the application name is "image processing method and device, electronic equipment and storage medium". The entirety of the application is incorporated by reference into this disclosure.

technical field

The present disclosure relates to the field of computer technology, and in particular to an image processing method, device, electronic equipment, storage medium and computer program product.

Background technique

At present, the beautification of the human face mainly focuses on the whitening of the whole face, the repair of blemishes, the shape adjustment of local organs, and the superimposition of makeup effects. It is still difficult to achieve the desired effect on some detailed makeup processing of local organs.

Contents of the invention

According to an aspect of the present disclosure, an image processing method is provided, including: determining a first mask image and a second mask image of a face image, the first mask image being used to characterize the face image A first part area, the second mask image is used to represent a second part area of the face image, the second part area is at least partially different from the first part area; lip processing information is obtained, and the The lip processing information includes area information and beautification parameter information, the area information is used to indicate the range of the second part area, and the beautification parameter information is used to indicate the beautification parameters of the second part area; according to the Adjusting the range of the second part area in the second mask image according to the area information and the first mask image; according to the beautification parameter information and the adjusted second mask image Beautification processing is performed to obtain a target image in which the color of the second part region is adjusted.

In a possible implementation manner, the adjusting the range of the second part region in the second mask image according to the region information and the first mask image includes: according to the region information and the first mask image A mask image, performing an image erosion operation on the second mask image to adjust the range of the second part area; during the image erosion process, the distance between the second part area and the first part area At least some of the regions overlap.

In a possible implementation manner, performing an image erosion operation on the second mask image according to the area information and the first mask image, and adjusting the range of the second part area includes: The first mask image performs image erosion from the first edge to the second edge of the second part area in the second mask image until the adjustment condition corresponding to the area information is satisfied.

In a possible implementation manner, the adjustment condition corresponding to the area information is that the ratio of the current second part area to the original second part area is the value of the area information.

In a possible implementation manner, the performing image erosion from the first edge to the second edge of the second part region in the second mask image according to the first mask image includes: using the first mask image The first edge of the second part area begins to determine the target pixel in turn toward the second edge; determine the target area according to the position of the target pixel in the second mask image; determine the target area according to the position of the first mask image in the target area The pixel value of each pixel in the second mask image is used to determine a reference pixel in the second mask image; and the pixel value of the target pixel is updated according to the pixel value of the reference pixel in the second mask image.

In a possible implementation manner, the determining the reference pixel in the second mask image according to the pixel value of each pixel in the target area in the first mask image includes: acquiring the first mask image A pixel value of each pixel in the target area in the mask image; determining a reference pixel in the second mask image according to a position in the first mask image of a pixel whose pixel value is greater than a pixel threshold.

In a possible implementation manner, the determining the first mask image of the face image includes: determining a first reference mask image, the first reference mask image is used to characterize the predicted shape of the first part region ; identifying a first part area in the face image; generating a first mask image according to the first part area and the first reference mask image.

In a possible implementation manner, the determining the second mask image of the face image includes: determining a second reference mask image, and the second reference mask image is used to characterize the edge of the first part region Predicting the shape; identifying a first part area in the face image; generating a second mask image according to the first part area and the second reference mask image.

In a possible implementation manner, the acquiring lip processing information includes: generating the region information in response to a trigger event for the region adjustment control; generating the region information in response to a trigger event for the color adjustment control the beautification parameter information; and determine the lip processing information according to the area information and the beautification parameter information.

In a possible implementation manner, the beautification process is performed on the face image according to the beautification parameter information and the adjusted second mask image to obtain the adjusted target of the second part region color The image includes: generating a beautification image according to the beautification parameter information and the adjusted second mask image, the beautification image is an image filled with the color represented by the beautification parameter information in the second part area; performing image fusion on the beautified image and the face image to obtain a target image in which the color of the second part region is adjusted.

In a possible implementation manner, the image processing method is used to add a lip line beautification effect to the face image, the first part area is a lip area, and the second part area is a lip line area.

According to an aspect of the present disclosure, there is provided an image processing apparatus, including: an image determination section configured to determine a first mask image and a second mask image of a face image, the first mask image being used for Characterizing a first part region of the human face image, the second mask image is used to characterize a second part region of the human face image, the second part region is at least partially different from the first part region; The information acquisition part is configured to acquire lip processing information, and the lip processing information includes area information and beautification parameter information; the range adjustment part is configured to adjust the lip processing information according to the area information and the first mask image. The range of the second part area in the second mask image; the color adjustment part is configured to perform beautification processing on the face image according to the beautification parameter information and the adjusted second mask image to obtain the obtained The target image whose color is adjusted for the second part area.

In a possible implementation manner, the range adjustment part includes: an image erosion subsection configured to perform an image erosion operation on the second mask image according to the region information and the first mask image , adjusting the range of the second part area; during the image erosion process, the second part area overlaps with at least part of the first part area.

In a possible implementation manner, the image erosion sub-part includes: an image erosion part configured to, according to the first mask image, generate the first edge of the second part region in the second mask image Image erosion is performed toward the second edge until the adjustment condition corresponding to the area information is satisfied.

In a possible implementation manner, the image erosion part includes: a target pixel determination subsection configured to sequentially determine target pixels from the first edge to the second edge of the second part area; the area determination subsection , configured to determine a target area according to the position of the target pixel in the second mask image; a reference pixel determining subsection, configured to determine each pixel in the target area according to the first mask image The pixel value of the second mask image determines the reference pixel in the second mask image; the pixel value updating subsection is configured to update the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image .

In a possible implementation manner, the reference pixel determination subsection includes: a pixel value acquisition subsection configured to acquire the pixel value of each pixel of the first mask image in the target area; the reference pixel The screening subpart is configured to determine a reference pixel in the second mask image according to the position of the pixel whose pixel value is greater than a pixel threshold in the first mask image.

In a possible implementation manner, the image determination part includes: a first reference image determination subsection configured to determine a first reference mask image, and the first reference mask image is used to characterize the first Predicting the shape of the part area; the first area identification subsection is configured to identify the first part area in the face image; the first image generation subsection is configured to combine the first part area with the first A first mask image is generated with reference to the mask image.

In a possible implementation manner, the image determination part includes: a second reference image determination subsection configured to determine a second reference mask image, and the second reference mask image is used to characterize the first The edge prediction shape of the part area; the second area identification subsection is configured to identify the first part area in the face image; the second image generation subsection is configured to combine the first part area with the The second reference mask image generates a second mask image.

In a possible implementation manner, the information acquisition part includes: an information determination subsection configured to generate the region information in response to a trigger event for the region adjustment control; A trigger event to generate the beautification parameter information; the information generating subpart is configured to determine the lip processing information according to the area information and the beautification parameter information.

In a possible implementation manner, the color adjustment part includes: a beautification image generating subsection configured to generate a beautification image according to the beautification parameter information and the adjusted second mask image, the beautification image In order to fill the image representing the color of the beautification parameter information in the second part area; the image fusion subsection is configured to perform image fusion on the beautification image and the face image to obtain the second part area The color-adjusted target image.

According to an aspect of the present disclosure, there is provided an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

According to one aspect of the present disclosure, a computer program product is provided, the computer program product includes a non-transitory computer storage medium storing a computer program, and the above method is implemented when the computer program is read and executed by a computer.

In the embodiment of the present disclosure, by identifying the first part area and the second part area of the face image, and adjusting the second part area based on the area information specified by the user and the first part area, the area that needs to be beautified and edited can be obtained. Landscaping. The beautification process can be used to add a lip line beautification effect to the face image, realizing the precise beautification effect on a specific area of the lips of the face image.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

FIG. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of a first mask image according to an embodiment of the present disclosure;

Fig. 3 shows a schematic diagram of a second mask image according to an embodiment of the present disclosure;

FIG. 4 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 5 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 6 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 7 shows a schematic diagram of a lip editing page according to an embodiment of the present disclosure;

FIG. 8 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 9 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 10 shows a schematic diagram of adjusting a second part area according to an embodiment of the present disclosure;

Fig. 11 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 12 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

Fig. 13 shows a schematic diagram of an image processing device according to an embodiment of the present disclosure;

Fig. 14 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure;

Fig. 15 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "at least one" herein means any one of a variety or any combination of at least two of a variety, for example, including at least one of A, B, and C, may mean including from A, B, and Any one or more elements selected from the set formed by C.

In addition, in order to better illustrate the present disclosure, numerous implementation details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without certain of the implementation details. In some embodiments, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the gist of the present disclosure.

In a possible implementation manner, the image processing method in the embodiment of the present disclosure may be executed by an electronic device such as a terminal device or a server. Wherein, the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, Any fixed or mobile terminal such as wearable devices. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the image processing method of the embodiments of the present disclosure by calling the computer-readable instructions stored in the memory by the processor.

FIG. 1 shows a flow chart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 1 , the image processing method of the embodiment of the present disclosure may include the following steps S101 to S104.

Step S101, determining a first mask image and a second mask image of a face image.

In a possible implementation manner, the face image is an image to be beautified, including a face to be processed. The electronic device can collect the face image through a built-in or connected image acquisition device, or collect it through other electronic devices and transmit it to the electronic device currently executing the image processing method for image processing. After acquiring the face image, the electronic device determines the first mask image and the second mask image of the face image. Wherein, the first mask image is used to represent the first part area of the face image, and the second mask image is used to represent the second part area of the face image. The second site area is at least partially different from the first site area, for example, the second site area may be within the first site area. The outer edge of the second part area can also coincide with the outer edge of the first part area. For example, the first part area may be the area where all the lips of the face to be processed are located in the face image, and the second part area may be the edge area of the area where all the lips of the face to be processed are located in the face image.

In a possible implementation manner, the first mask image and the second mask image may be binary images, the pixel values in the first part area and the second part area are 1, and the pixel values in other areas are 0. In order to further perform image processing on each pixel value in the first part area and the second part area by using different processing intensities, so as to improve the image processing effect. Each pixel value in the first mask image and the second mask image can also be a value between 0 and 1, which is used to distinguish the first part area or the second part area from other areas, and to characterize the subsequent image processing process processing intensity. In the embodiment of the present disclosure, the processing intensity represents the corresponding beautification strength. Exemplarily, when the beautification parameter used for image processing is N, the pixel value can be beautified by multiplying each pixel value by the beautification parameter N, for example, by using the beautification parameter N to process a pixel value with a pixel value of 1, Pixel values with a pixel value of 0.5 are processed by 0.5 times the beautification parameter N. The electronic device can determine the first part area and the second part area according to the pixel value of each pixel and a preset pixel threshold. Take the preset pixel threshold value of 0.5 as an example for illustration. In the first mask image, the positions of the pixels whose pixel values are not less than 0.5 are located in the first part area, and the positions of the pixels whose pixel values are less than 0.5 are located in other areas than the first part area. In the second mask image, the pixel values not less than 0.5 are located in the second part area, and the pixel values smaller than 0.5 are located in other areas than the second part area.

Fig. 2 shows a schematic diagram of a first mask image according to an embodiment of the disclosure. As shown in FIG. 2 , the first mask image 200 is the same size as the acquired face image, and includes a first region 201 composed of pixels whose pixel values are not less than the pixel threshold, and a first region 201 composed of pixels whose pixel values are less than the pixel threshold. The second area 202 of . Wherein, the first area 201 corresponds to the lip position in the face image, that is, the position of the lips representing the face in the face image. The second area 202 corresponds to other positions in the human face image except for the lip position, and represents other positions in the human face image except for the position of the lips of the human face. That is to say, the first area 201 is the first part area represented by the first mask image.

Fig. 3 shows a schematic diagram of a second mask image according to an embodiment of the present disclosure. As shown in FIG. 3 , the second mask image 300 is the same size as the acquired face image, and includes a third area 301 composed of pixels whose pixel values are not less than the pixel threshold, and includes pixels whose pixel values are less than the pixel threshold. The fourth region 302 of . Wherein, the third area 301 corresponds to the lip edge position in the face image, that is, the position where the lip line needs to be drawn to represent the lips of the face in the face image. The fourth area 302 corresponds to positions other than lip edge positions in the face image, and represents positions in the face image where lip lines do not need to be drawn. That is to say, the third area 301 is the second part area represented by the second mask image. In a possible implementation manner, the second site region has an inner edge 303 and an outer edge 304 .

In a possible implementation manner, the first mask image and the second mask image may be determined according to a predetermined standard material. FIG. 4 shows a flow chart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 4 , the process of determining the first mask image of the face image in step S101 may include the following steps S401 to S403.

Step S401. Determine a first reference mask image, which is used to characterize the predicted shape of the first part region.

Step S402, identifying the first part area in the face image.

Step S403, generating a first mask image according to the first part region and the first reference mask image.

FIG. 5 shows a flow chart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 5 , the process of determining the second mask image of the face image in step S101 may include the following steps S501 to S503.

Step S501. Determine a second reference mask image, which is used to characterize the edge prediction shape of the first part region.

Step S502, identifying the first part area in the face image.

Step S503, generating a second mask image according to the first part region and the second reference mask image.

In a possible implementation manner, the first reference mask image and/or the second reference mask image may be a mask image material preset according to the simulated face, or a mask uploaded by the user according to his or her own needs. Image footage. When the image processing method of the embodiment of the present disclosure is used to add a lip line beautification effect to the face image, the first part area is the lip area, and the second part area is the lip line area, the first reference mask image can be the user The image uploaded or generated by the electronic device representing the shape of the lips, and the second reference mask image may be an image uploaded by the user or generated by the electronic device representing the shape of the lip line.

In a possible implementation manner, the method of recognizing the face image to obtain the first part area is key point recognition, and the obtained first part area includes a plurality of key position points, such as the position of the corner of the mouth, the position of the upper edge, the position of the lower Edge position and lip bead position etc. The determined first reference mask image and the second reference mask image also include a plurality of key position points, and the first mask can be determined by mapping key position points between the first reference mask image and the first part area image, the second mask image is determined by performing key position point mapping between the second reference mask image and the first part region. Based on the key point mapping, the first mask image and the second mask image determined in the above manner can accurately locate the positions of the first part area and the second part area in the face image.

Step S102, acquiring lip treatment information.

In a possible implementation manner, the lip processing information may include area information and beautification parameter information, the area information is used to indicate the area range of the second part to be beautified in the face image, and the beautification parameter information is used to indicate the face image Beautification parameters of the second part area to be beautified. The beautification parameters may include any parameters for beautifying the lips, such as color, brightness, contrast, and transparency. The lip treatment information can be acquired in any way, such as directly receiving the lip treatment information sent by other electronic devices, or determining with the user through human-computer interaction.

Fig. 6 shows a flowchart of an image processing method according to an embodiment of the present disclosure, as shown in Fig. 6 . In a possible implementation manner, in an application scenario where the lip treatment information is determined through human-computer interaction with the user, step S102 may be implemented by the electronic device through steps S601 to S603.

Step S601 , generating area information in response to a trigger event for the area adjustment control.

Step S602, generating beautification parameter information in response to a trigger event for the color adjustment control.

Step S603, determining lip processing information according to the area information and beautification parameter information.

FIG. 7 shows a schematic diagram of a lip editing page 700 according to an embodiment of the present disclosure. In the case that the electronic device is a terminal device, the lip editing page 700 may be displayed by a display device built in or connected to the electronic device. In the case where the electronic device is a server, the lip editing page 700 may be sent to the terminal device through the server, and displayed by a display device of the terminal device. As shown in FIG. 7 , the lip editing page 700 includes a color adjustment control 701 and an area adjustment control 702 .

In a possible implementation, the color adjustment control 701 can include a preset color control that can be directly selected by the user through human-computer interaction, and can be dragged or input by the user through human-computer interaction. Autocolor controls (R, G, B) that determine the color by the color value. During the human-computer interaction process, the electronic device may receive a preset color from among the preset color controls Color 1 to Color 6 that is directly selected by the user by clicking. Alternatively, the RGB value determined by the user within 0 to 255 by inputting or dragging may also be received to obtain the desired color.

In a possible implementation manner, the color adjustment control 701 further includes an intensity adjustment control (L), which is used to represent the intensity of the color to be adjusted, and the selected value can be between 0 and 1. The beautification parameter information may be determined according to the color and color intensity determined or selected by the user through human-computer interaction.

In a possible implementation manner, the area adjustment control 702 may include a drag bar that the user can drag to determine a width adjustment ratio through human-computer interaction, and an input box that can input a desired adjustment width ratio. In a possible implementation manner, the adjusted width ratio determined by the user may be a numerical value, which is used to represent the ratio of the required width of the second part region to the current second part region. The electronic device can generate area information according to the adjustment width ratio determined by the user through human-computer interaction. In a possible implementation manner, the electronic device may generate lip treatment information including area information and beautification parameter information after the user triggers color adjustment control 701 to generate beautification parameter information and triggers area adjustment control 702 to generate area information.

In a possible implementation, the lip editing page 700 can also be used to display the face image 703 (blocking part of the area), and the second part area 704 in the face image 703, which is determined by the electronic device After the lip processing information, the image is beautified according to the lip processing information and displayed to the user in real time. If the user is not satisfied, he can re-determine the new lip treatment information through human-computer interaction.

In the embodiment of the present disclosure, based on the area adjustment control and color adjustment control, the user can adjust the area information and beautification parameter information according to his own interests, so as to obtain the required lip processing information, and then improve the lip line beautification effect.

Step S103, adjusting the range of the second part area in the second mask image according to the area information and the first mask image.

In a possible implementation manner, the second mask image may be adjusted based on the first mask image and the region information determined by the user, so as to modify the range of the second part region represented in the second mask image. Before and after the adjustment, the position of the second part region represented by the second mask image is within the first part region represented by the first mask image. The adjustment process of the second mask image can be realized by image erosion, which is used to shrink or refine objects in the image.

Fig. 8 shows a flow chart of an image processing method according to an embodiment of the present disclosure. As shown in Fig. 8, step S103 in Fig. 1 can be updated to step S801, and step S104 can be updated to step S802, which will be shown in conjunction with Fig. 8 Outlined steps are described.

Step S801, according to the area information and the first mask image, perform an image erosion operation on the second mask image to adjust the range of the second part area; during the image erosion process, the second part area can be at least the same as the first part area Some areas overlap.

In a possible implementation manner, during the image erosion process, both the area information and the first mask image are used to limit the image erosion range. Wherein, the area information may be used to define the adjusted area width of the second part. Exemplarily, the second part area is a continuous area including the first edge and the second edge, and the direction of image erosion may be from the first edge to the second edge.

Step S802, according to the first mask image, perform image erosion from the first edge to the second edge of the second part area in the second mask image until the adjustment condition corresponding to the area information is satisfied.

In a possible implementation manner, as shown in FIG. 3 , the first edge of the second part area (third area 301) is the inner edge 303 located on the inner side, and the second edge of the second part area is the outer edge 303 located on the outer side. Edge 304.

In a possible implementation manner, when the area information is a numerical value, the adjustment condition corresponding to the area information may be that the ratio of the current second part area to the original second part area is the value of the area information.

That is to say, the preset lip edge area can be reduced by adjusting the area information. For example, in the case where the preset area width of the second part is 5 mm and the area information is 0.3, the width of the second part area obtained after image erosion is 1.5 mm.

In a possible implementation, during the image erosion process, the first mask image is used to filter out the reference pixels used for image erosion in the second mask image, and then implement the image erosion process based on the filtered reference pixels .

In the embodiment of the present disclosure, since the image erosion operation is performed on the second mask image based on the area and the first mask image, the range of the second part area can be adjusted; and according to the first mask image, the second mask image Performing image erosion from the first edge of the second part area in the image to the second edge can improve the consistency between the obtained beautified and edited area outline and the first area outline, thereby improving the lip line beautification effect.

Fig. 9 shows a flowchart of an image processing method according to an embodiment of the present disclosure, as shown in Fig. 9 . In step S802, the process of performing image erosion from the first edge to the second edge of the second part area in the second mask image according to the first mask image can be updated to steps S901 to S904, which will be performed in conjunction with the steps shown in FIG. 9 illustrate.

Step S901 , sequentially determine target pixels from the first edge to the second edge of the second part area.

Step S902. Determine the target area according to the position of the target pixel in the second mask image.

Step S903, determining a reference pixel in the second mask image according to the pixel value of each pixel in the target area in the first mask image.

Step S904, updating the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image.

In a possible implementation manner, the minimum pixel value in each reference pixel may be compared, so as to replace the pixel value of the target pixel with the minimum pixel value in each reference pixel in the second mask image.

In a possible implementation manner, the selection of the target pixels may be determined in an outward order from the first edge of the second part area position. After the target pixel is replaced each time, a new target pixel can be re-determined in the second part area, and the image erosion is performed again until the image erosion process is completed. The size of the target area may be a predetermined preset size, and after each determination of the target pixel, an area whose center is the target pixel and whose size is the preset size is determined as the corresponding target area. For example, when the preset size is 3×3 and the coordinates of the target pixel are (2, 4), the target area consists of pixels (1, 3), (1, 4), (1, 5), (2, 3 ), (2,4), (2,5), (3,3), (3,4) and (3,5).

In a possible implementation manner, after the target area is determined, the reference pixel is obtained by filtering according to the pixel value of each pixel in the target area in the first mask image. The screening method can be implemented based on a preset pixel threshold, that is, the above-mentioned update of the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image can be implemented based on steps S9041 to S9042.

Step S9041, obtaining the pixel value of each pixel of the first mask image in the target area;

Step S9042, determine in the second mask image according to the position of the pixel whose pixel value is greater than the pixel threshold in the first mask image.

In a possible implementation, the target area is 3×3, and the pixel coordinates included are (1, 3), (1, 4), (1, 5), (2, 3), (2 , 4), (2, 5), (3, 3), (3, 4) and (3, 5) as examples for illustration. When the coordinates in the first mask image are (1,3), (1,4), (1,5), (2,3), (2,4), (2,5), (3,3) , (3, 4) and (3, 5) are 1, 1, 0.7, 1, 0.8, 0.1, 0.9, 0.2 and 0 respectively, and when the preset pixel threshold is 0.5, the filtered The coordinates of the reference pixels are (1, 3), (1, 4), (1, 5), (2, 3), (2, 4) and (3, 3), respectively. When the pixel values in the above reference pixels (1,3), (1,4), (1,5), (2,3), (2,4) and (3,3) in the second mask image When they are 0.1, 0.2, 1, 0.7, 0.8, 0, and 0.9, obtain the minimum value 0 to replace the pixel value 0.8 in the target pixel (2, 4).

Fig. 10 shows a schematic diagram of adjusting a second part area according to an embodiment of the present disclosure. As shown in FIG. 10 , after determining the target area 1001 centered on the current target pixel in the second mask image, a target area 1002 at the position of the target area in the first mask image is also obtained. The reference pixels are obtained by screening the pixel values of each pixel in the target area 1002 in the first mask image, and an image erosion area 1003 composed of a plurality of reference pixels in the target area 1001 in the second mask image is obtained. The pixel value in the target pixel is replaced by the minimum value 0 in the image erosion area 1003 .

After image erosion is completed according to the region information and the first mask image, a second mask image after adjusting the second part region represented therein is obtained.

Step S104: Perform beautification processing on the human face image according to the beautification parameter information and the adjusted second mask image to obtain a target image in which the color of the second part region is adjusted.

In a possible implementation, after the second mask image is adjusted according to the area information and the first mask image, according to the beautification parameter information in the beautification processing information and the adjusted second mask image in the face image Beautify the second part of the region to obtain the target image.

FIG. 11 shows a flowchart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 11 , step S104 in FIG. 1 can be updated to steps S1101 to S1102 , which will be described in conjunction with the steps shown in FIG. 11 .

Step S1101, generate a beautification image according to the beautification parameter information and the adjusted second mask image, the beautification image is an image filled with the color represented by the beautification parameter information in the second part area.

In a possible implementation, the beautified image may be an image with the same size as the face image, wherein only the second part area is filled with the beautification parameter information represented by the intensity of each pixel representing the second part area in the second mask image. color, and set the other areas to be transparent, that is, set the pixel value to 0.

Step S1102, performing image fusion on the beautified image and the face image to obtain the target image whose color is adjusted in the second part area.

In a possible implementation manner, the image fusion manner may be any fusion manner, such as any manner such as multiplication and transparency fusion. Wherein, the multiplication process is to superimpose the position of the second part region in the beautified image on the position of the second part region in the face image in the form of a slideshow.

The following describes the application of the image processing method provided by the embodiment of the present disclosure in an actual scene, taking the lip line beautification image processing scene based on real-time collected face images as an example.

FIG. 12 shows a flowchart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 12 , the image processing method of the embodiment of the present disclosure may include the following steps S1201 to S1203.

Step S1201, according to the texture of the lip line Mask, acquire the image of the part where the lip line needs to be done.

Step S1202 , perform image erosion processing on the lip line Mask texture according to the input lip line width ratio.

In a possible implementation, the lip line Mask texture (corresponding to the second part area of the second mask image in the above embodiment) is used for texture sampling of the kernel area (the square area surrounding the current pixel), which can be obtained through the full The lip Mask texture (corresponding to the first part area of the first mask image in the above embodiment) determines the pixels that need to be sampled and the pixels that do not need to be sampled in the kernel area. Exemplarily, the pixels beyond the lip area in the full lip Mask texture are not sampled, so that the outer circle of the lip line Mask texture does not change, that is, the outer circle of the lip line Mask texture is not corroded, but only The inner ring of the lip mask texture is etched.

In a possible implementation, the minimum value of the sampling result of the entire kernel area is used as the output value of the current pixel, and the pixel value of the current pixel is updated, so as to determine the lip line area where the lip line effect is to be performed (corresponding to the above-mentioned embodiment) The second part area obtained after performing image erosion).

Step S1203, based on the lip line intensity selected by the user, fuse the input target color with the original pixel color.

In a possible implementation manner, the input target color and the original pixel color (lip line area) may be fused by using the multiply blending mode to obtain a lip line effect image (corresponding to the target image in the above embodiment).

In the embodiment of the present disclosure, the lip line Mask texture is used to obtain the image of the part where the lip line needs to be made, and then the image erosion process is performed on the lip line Mask texture to obtain a lip line area with a suitable width; at the same time, based on the lip line strength selected by the user , to blend the target color with the original pixel color to obtain more styles of lip line effect images.

The image processing method of the embodiment of the present disclosure may be used to beautify the lips of a human face in a human face image. For example, a face image may be acquired by an electronic device, and the feature points of the face in the face image may be recognized to obtain the lip region of the face. According to the preset or user uploaded first reference mask image and the lip area, the first mask image representing the position of the human face lip area in the face image is obtained, and according to the preset or user uploaded second reference The mask image and the lip region obtain a second mask image representing the position of the lip edge of the human face in the human face image. At the same time, the electronic device determines the area information indicating the area where the user wishes to increase the lip line effect and the beautification parameter information indicating the color, intensity and brightness of the lip line that the user wishes to increase through human-computer interaction with the user. Adjust the second mask image according to the area information and the first mask image to determine the final area where the lip line effect needs to be increased, and obtain the lip line effect for beautification according to the beautification parameter information and the adjusted second mask image The beautified image of the area, the beautified image and the face image are multiplied to obtain the face image with the lip line effect after beautification.

The embodiment of the present disclosure can identify the first part area and the second part area of the face image, and adjust the second part area based on the area information specified by the user and the first part area, so as to obtain the area that needs to be beautified and perform beautification processing. The effect of precisely beautifying the specific area of the lip of the face image is realized.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method in the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, the present disclosure also provides an image processing device, electronic equipment, computer-readable storage medium, and program, all of which can be used to implement any image processing method provided in the present disclosure. For corresponding technical solutions and descriptions, please refer to the method section corresponding record.

Fig. 13 shows a schematic diagram of an image processing device according to an embodiment of the present disclosure. As shown in FIG. 13 , the image processing apparatus of the embodiment of the present disclosure includes an image determination section 1301 , an information acquisition section 1302 , a range adjustment section 1303 , and a color adjustment section 1304 .

The image determining part 1301 is configured to determine a first mask image and a second mask image of the face image, the first mask image is used to characterize the first part region of the face image, and the second mask image The mask image is used to represent a second part region of the human face image, the second part region is at least partially different from the first part region;

The information acquiring part 1302 is configured to acquire lip processing information, and the lip processing information includes area information and beautification parameter information;

The range adjustment part 1303 is configured to adjust the range of the second part area in the second mask image according to the area information and the first mask image;

The color adjustment part 1304 is configured to perform beautification processing on the face image according to the beautification parameter information and the adjusted second mask image, so as to obtain the target image in which the color of the second part region is adjusted.

In a possible implementation manner, the range adjustment part 1303 includes:

The image erosion sub-part is configured to perform an image erosion operation on the second mask image according to the area information and the first mask image, and adjust the range of the second part area; During the process, the second part area overlaps with at least part of the first part area.

In a possible implementation manner, the image erosion subsection includes:

The image erosion part is configured to perform image erosion from the first edge to the second edge of the second part area in the second mask image according to the first mask image until the adjustment corresponding to the area information is satisfied. condition.

In a possible implementation manner, the image erosion part includes:

The target pixel determination subsection is configured to sequentially determine target pixels from the first edge to the second edge of the second part area;

an area determining subsection configured to determine a target area according to the position of the target pixel in the second mask image;

a reference pixel determining subsection configured to determine a reference pixel in the second mask image according to a pixel value of each pixel in the target area in the first mask image;

The pixel value updating subsection is configured to update the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image.

In a possible implementation manner, the reference pixel determining subsection includes:

a pixel value obtaining subsection configured to obtain a pixel value of each pixel of the first mask image in the target area;

The reference pixel screening subsection is configured to determine a reference pixel in the second mask image according to the position of the pixel whose pixel value is greater than a pixel threshold in the first mask image.

In a possible implementation manner, the image determining part 1301 includes:

The first reference image determination subpart is configured to determine a first reference mask image, the first reference mask image is used to characterize the predicted shape of the first part region;

The first area identification subpart is configured to identify a first part area in the face image;

The first image generating subsection is configured to generate a first mask image according to the first part region and the first reference mask image.

In a possible implementation manner, the image determining part 1301 includes:

The second reference image determination subpart is configured to determine a second reference mask image, the second reference mask image is used to characterize the edge prediction shape of the first part region;

The second area identification subpart is configured to identify the first part area in the face image;

The second image generating subsection is configured to generate a second mask image according to the first part region and the second reference mask image.

In a possible implementation manner, the information acquiring part 1302 includes:

The information determination subpart is configured to generate the area information in response to a trigger event for the area adjustment control; generate the beautification parameter information in response to a trigger event for the color adjustment control;

The information generation subpart is configured to determine the lip treatment information according to the area information and the beautification parameter information.

In a possible implementation manner, the color adjustment part 1304 includes:

The beautification image generation subpart is configured to generate a beautification image according to the beautification parameter information and the adjusted second mask image, the beautification image is a representation of filling the beautification parameter information in the second part area color image;

The image fusion subpart is configured to perform image fusion on the beautification image and the face image to obtain a target image in which the color of the second part region is adjusted.

In some embodiments, the functions or parts included in the apparatus provided by the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments, and for specific implementation, refer to the descriptions of the above method embodiments.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, where the computer-readable codes are stored in a processor of an electronic device In the case of running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

FIG. 14 shows a schematic diagram of an electronic device 1400 according to an embodiment of the present disclosure. For example, the electronic device 1400 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 14 , electronic device 1400 may include one or more of the following components: processing component 1401, memory 1402, power supply component 1403, multimedia component 1404, audio component 1405, input/output (Input/Ouput, I/O) interface 1406, sensor component 1407, and communication component 1408.

The processing component 1401 generally controls the overall operations of the electronic device 1400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1401 may include one or more processors 1409 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 1401 may include one or more components that facilitate interaction between processing component 1401 and other components. For example, processing component 1401 may include a multimedia portion to facilitate interaction between multimedia component 1404 and processing component 1401 .

The memory 1402 is configured to store various types of data to support operations at the electronic device 1400 . Examples of such data include instructions for any application or method operating on the electronic device 1400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1402 can be realized by any type of volatile or non-volatile memory device or their combination, such as Static Random-Access Memory (Static Random-Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically -Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM), Programmable Read-Only Memory (Programmable read-only memory, PROM), Read Only Memory (Read -Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 1403 provides power to various components of the electronic device 1400 . Power components 1403 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 1400 .

The multimedia component 1404 includes a screen providing an output interface between the electronic device 1400 and the user. In some embodiments, the screen may include a liquid crystal display (Liquid Crystal Display, LCD) and a touch panel (Touch Panel, TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 1404 includes at least one of the following: a front camera and a rear camera. When the electronic device 1400 is in an operation mode, such as a shooting mode or a video mode, at least one of the front camera and the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 1405 is configured to output and input audio signals, or to output audio signals, or to input audio signals. For example, the audio component 1405 includes a microphone (Microphone, MIC), which is configured to receive external audio signals when the electronic device 1400 is in operation modes, such as calling mode, recording mode and voice recognition mode. Received audio signals may be stored in memory 1402 or sent via communication component 1408 . In some embodiments, the audio component 1405 also includes a speaker configured to output audio signals.

The I/O interface 1406 provides an interface between the processing component 1401 and the peripheral interface part, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 1407 includes one or more sensors for providing various aspects of status assessment for electronic device 1400 . For example, the sensor component 1407 can detect the open/closed state of the electronic device 1400, the relative positioning of components, for example, the components are the display and the keypad of the electronic device 1400, the sensor component 1407 can also detect the electronic device 1400 or one of the electronic devices 1400 Changes in position of components, presence or absence of user contact with electronic device 1400 , electronic device 1400 orientation or acceleration/deceleration and temperature changes in electronic device 1400 . The sensor assembly 1407 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 1407 may also include an optical sensor, such as a Complementary Metal-Oxide-Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 1407 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 1408 is configured to facilitate wired or wireless communication between the electronic device 1400 and other devices. The electronic device 1400 can access a wireless network based on a communication standard, such as a wireless network (Wireless Network), a second generation mobile communication technology (Second Generation, 2G) or a third generation mobile communication technology (Third Generation, 3G), or their combination. In an exemplary embodiment, the communication component 1408 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1408 also includes a Near Field Communication (NFC) portion to facilitate short-range communication. For example, the NFC part can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (Infrared Data Association, IrDA) technology, Ultra Wide Band (Ultra Wide Band, UWB) technology, Bluetooth (Bluetooth, BT) technology and other technology to achieve.

In an exemplary embodiment, the electronic device 1400 may be implemented by one or more Application Specific Integrated Circuits (Application Specific Integrated Circuit, ASIC), Digital Signal Processor (Digital Signal Processor, DSP), Digital Signal Processing Device (Digital Signal Processing Device , DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components to implement the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 1402 including computer program instructions, which can be executed by the processor 1409 of the electronic device 1400 to implement the above method.

This disclosure relates to the field of computer technology. By acquiring the image information of the target object in the real environment, and then using various visual correlation algorithms to detect or identify the relevant characteristics, states and attributes of the target object, so as to obtain the image information that matches the specific application. AR effect combining virtual and reality. Exemplarily, the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places. Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc. Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display. The relevant characteristics, states and attributes of the target object can be detected or identified through the convolutional neural network. The above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.

FIG. 15 shows a schematic diagram of another electronic device 1500 according to an embodiment of the present disclosure. For example, the electronic device 1500 may be provided as a server. Referring to FIG. 15 , an electronic device 1500 includes a processing component 1501 , and also includes one or more processors, and a memory resource represented by a memory 1502 configured to store instructions executable by the processing component 1501 , such as application programs. An application program stored in memory 1502 may include one or more portions each corresponding to a set of instructions. In addition, the processing component 1501 is configured to execute instructions to perform the above method.

The electronic device 1500 may also include a power component 1503 configured to perform power management of the electronic device 1500 , a wired or wireless network interface 1504 configured to connect the electronic device 1500 to a network, and an I/O interface 1505 . Electronic device 1500 can operate based on the operating system stored in memory 1502, such as Microsoft server operating system (Windows Server™), based on graphical user interface operating system (Mac OS X™), multi-user multi-process computer operating system (Unix™), free and An open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as a memory 1502 including computer program instructions, which can be executed by the processing component 1501 of the electronic device 1500 to implement the above method.

The present disclosure includes at least one of the following: a system, a method and a computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, Random-Access Memory (RAM), Read-Only Memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM) or flash memory, Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compression Disk Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM ), Digital Versatile Disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as at least one of the Internet, a local area network, a wide area network, and a wireless network . The network may include at least one of: copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, Instruction Set Architectures (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in the form of one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays (Field Programmable Gate Array, FPGA) or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuit can execute computer-readable program instructions, thereby implementing various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to at least one of flowchart illustrations and block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of at least one of the flowchart illustrations and block diagrams, and combinations of blocks in the flowchart illustrations and block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause at least one of computers, programmable data processing devices and other devices to work in a specific way, so that the computer-readable The medium then includes an article of manufacture including instructions for implementing various aspects of the functions/acts specified in at least one block in the flowcharts and block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in at least one block in the flowcharts and block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a portion, a program segment, or a portion of an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also to be noted that each block of at least one of the block diagrams and flowcharts, and combinations of blocks of at least one of the block diagrams and flowcharts, may be implemented with a dedicated hardware-based computer that performs specified functions or actions. system, or it may be implemented by a combination of special purpose hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.

The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, and the same or similar points can be referred to each other.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

If the disclosed technical solution involves personal information, the products applying the disclosed technical solution have clearly notified the personal information processing rules and obtained the individual's independent consent before processing personal information. If the disclosed technical solution involves sensitive personal information, the products applying the disclosed technical solution have obtained individual consent before processing sensitive personal information, and at the same time meet the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set up to inform that it has entered the scope of personal information collection, and personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed to agree to the collection of his personal information; or On the personal information processing device, when the personal information processing rules are informed with obvious signs/information, personal authorization is obtained through pop-up information or by asking individuals to upload their personal information; among them, the personal information processing rules may include Information such as the information processor, the purpose of personal information processing, the method of processing, and the type of personal information processed.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terminology used in this disclosure is intended to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed in this disclosure .

Industrial Applicability

Claims

An image processing method, the method is executed by an electronic device, the method comprising:

Determining a first mask image and a second mask image of the face image, the first mask image is used to characterize the first part region of the face image, and the second mask image is used to characterize the a second part area of a human face image, said second part area being at least partially different from said first part area;

Obtain lip processing information, the lip processing information includes area information and beautification parameter information, the area information is used to indicate the range of the second part area, and the beautification parameter information is used to indicate the second part Beautification parameters of the area;

adjusting the range of the second part region in the second mask image according to the region information and the first mask image;

Perform beautification processing on the human face image according to the beautification parameter information and the adjusted second mask image to obtain a target image in which the color of the second part region is adjusted.
The method according to claim 1, wherein said adjusting the range of the second part region in the second mask image according to the region information and the first mask image comprises:

According to the area information and the first mask image, perform an image erosion operation on the second mask image to adjust the range of the second part area; during the image erosion process, the second part The region coincides with at least part of the region of the first site.
The method according to claim 2, wherein the image erosion operation is performed on the second mask image according to the area information and the first mask image, and adjusting the range of the second part area includes :

According to the first mask image, performing image erosion from the first edge to the second edge of the second part area in the second mask image until the adjustment condition corresponding to the area information is satisfied.
The method according to claim 3, wherein the adjustment condition corresponding to the area information is that the ratio of the current second part area to the original second part area is the value of the area information.
The method according to claim 3 or 4, wherein said performing image erosion from the first edge to the second edge of the second part region in the second mask image according to the first mask image comprises:

Determining target pixels sequentially from the first edge of the second part area to the second edge;

determining a target area according to the position of the target pixel in the second mask image;

determining a reference pixel in the second mask image according to the pixel value of each pixel in the target area in the first mask image;

updating the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image.
The method according to claim 5, wherein said determining the reference pixel in the second mask image according to the pixel value of each pixel in the target area in the first mask image comprises:

Acquiring the pixel value of each pixel of the first mask image in the target area;

Determining a reference pixel in the second mask image according to the position of the pixel whose pixel value is greater than a pixel threshold in the first mask image.
The method according to any one of claims 1 to 6, wherein said determining the first mask image of the face image comprises:

determining a first reference mask image, the first reference mask image being used to characterize the predicted shape of the first part region;

identifying a first part region in the face image;

A first mask image is generated according to the first part region and the first reference mask image.
The method according to any one of claims 1 to 7, wherein said determining the second mask image of the face image comprises:

determining a second reference mask image, the second reference mask image being used to characterize the edge prediction shape of the first part region;

identifying a first part region in the face image;

A second mask image is generated according to the first part region and the second reference mask image.
The method according to any one of claims 1 to 8, wherein said obtaining lip treatment information comprises:

generating the area information in response to a trigger event for the area adjustment control;

generating the beautification parameter information in response to a trigger event for the color adjustment control;

The lip processing information is determined according to the area information and the beautification parameter information.
The method according to any one of claims 1 to 9, wherein the face image is beautified according to the beautification parameter information and the adjusted second mask image to obtain the first The target image whose color is adjusted in the second part area includes:

generating a beautification image according to the beautification parameter information and the adjusted second mask image, where the beautification image is an image filled with a color represented by the beautification parameter information in the second part region;

Image fusion is performed on the beautified image and the face image to obtain a target image in which the color of the second part region is adjusted.
The method according to any one of claims 1 to 10, wherein the image processing method is used to add a lip line beautification effect to the face image, the first part area is a lip area, and the second The second part area is the lip line area.
An image processing device, the device comprising:

The image determination part is configured to determine a first mask image and a second mask image of the face image, the first mask image is used to characterize the first part area of the face image, and the second mask The code image is used to represent a second part area of the human face image, the second part area is at least partially different from the first part area;

The information acquisition part is configured to acquire lip processing information, and the lip processing information includes area information and beautification parameter information;

a range adjustment part configured to adjust the range of the second part region in the second mask image according to the region information and the first mask image;

The color adjustment part is configured to perform beautification processing on the human face image according to the beautification parameter information and the adjusted second mask image, so as to obtain a target image in which the color of the second part region is adjusted.
The device according to claim 12, wherein the range adjustment part is further configured to perform an image erosion operation on the second mask image according to the area information and the first mask image, and adjust the The range of the second part area; during the image erosion process, the second part area overlaps with at least part of the first part area.
The device according to claim 13, wherein the range adjustment part is further configured to, according to the first mask image, move from the first edge of the second part region in the second mask image to the second Image erosion is performed on the edge until the adjustment condition corresponding to the area information is met.
The device according to claim 14, wherein the adjustment condition corresponding to the area information is that the ratio of the current second part area to the original second part area is the value of the area information.
The device according to claim 14 or 15, wherein the range adjustment part is further configured to sequentially determine target pixels from the first edge to the second edge of the second part area;

determining a target area according to the position of the target pixel in the second mask image;

determining a reference pixel in the second mask image according to the pixel value of each pixel in the target area in the first mask image;

updating the pixel value of the target pixel according to the pixel value of the reference pixel in the second mask image.
The device according to claim 16, wherein the range adjustment part is further configured to obtain the pixel value of each pixel in the target area of the first mask image;

Determining a reference pixel in the second mask image according to the position of the pixel whose pixel value is greater than a pixel threshold in the first mask image.
The device according to any one of claims 12 to 17, wherein the image determination part is further configured to determine a first reference mask image, the first reference mask image is used to characterize the first part area prediction shape;

identifying a first part region in the face image;

A first mask image is generated according to the first part region and the first reference mask image.
The device according to any one of claims 12 to 18, wherein the image determining part is further configured to determine a second reference mask image, the second reference mask image is used to characterize the first part The edge prediction shape of the region;

identifying a first part region in the face image;

A second mask image is generated according to the first part region and the second reference mask image.
The device according to any one of claims 12 to 19, wherein the information acquiring part is further configured to generate the area information in response to a trigger event for the area adjustment control;

generating the beautification parameter information in response to a trigger event for the color adjustment control;

The lip processing information is determined according to the area information and the beautification parameter information.
The device according to any one of claims 12 to 20, wherein the color adjustment part is further configured to generate a beautified image according to the beautification parameter information and the adjusted second mask image, the beautification The image is an image filled with the representation color of the beautification parameter information in the second part area;

Image fusion is performed on the beautified image and the face image to obtain a target image in which the color of the second part region is adjusted.
The device according to any one of claims 12 to 21, wherein the image processing device is configured to add a lip line beautification effect to the face image, the first part area is a lip area, and the second The part area is the lip line area.
An electronic device comprising:

processor;

memory configured to store processor-executable instructions;

Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-11.
A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.
A computer program product, the computer program product comprising a non-transitory computer storage medium storing a computer program, when the computer program is read and executed by a computer, the method according to any one of claims 1 to 11 is implemented.