WO2023025239A1 - 人像唇部的自动上妆方法、装置、设备、存储介质及程序产品 - Google Patents

人像唇部的自动上妆方法、装置、设备、存储介质及程序产品 Download PDF

Info

Publication number
WO2023025239A1
WO2023025239A1 PCT/CN2022/114755 CN2022114755W WO2023025239A1 WO 2023025239 A1 WO2023025239 A1 WO 2023025239A1 CN 2022114755 W CN2022114755 W CN 2022114755W WO 2023025239 A1 WO2023025239 A1 WO 2023025239A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
portrait
lip
lips
key points
Prior art date
Application number
PCT/CN2022/114755
Other languages
English (en)
French (fr)
Inventor
马晓
陈望都
王�琦
潘兴浩
李康敬
Original Assignee
咪咕视讯科技有限公司
咪咕文化科技有限公司
中国移动通信集团有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 咪咕视讯科技有限公司, 咪咕文化科技有限公司, 中国移动通信集团有限公司 filed Critical 咪咕视讯科技有限公司
Publication of WO2023025239A1 publication Critical patent/WO2023025239A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the fields of audio, video and image processing, and in particular to a method, device, equipment, storage medium and program product for automatically applying makeup to the lips of a portrait.
  • the smart terminal After the smart terminal collects portrait information through the camera, it can automatically apply makeup to the collected portrait according to the user's choice.
  • This function has been widely used in the smart terminal's own camera and video applications, and Processing of portraits collected in real time during Internet live broadcast.
  • the outline of the lips is first determined through the portrait, and then the color is applied according to the lipstick number selected by the user.
  • the portrait especially the facial area
  • the related technology performs curve fitting and coloring on the basis of the key points of face detection, and does not adjust the position of the key points.
  • the position of the key points is not accurate and does not match the actual position, which has a great impact on the makeup effect.
  • the embodiment of the present disclosure provides a method for automatically applying makeup to the lips of a portrait, the method comprising:
  • Extract lip key points from the portrait facial image and detect the portrait face orientation and lip shape in the portrait facial image, and adjust the position of the lip key point according to the portrait facial orientation and lip shape;
  • Fusion coloring is performed using the target lipstick color according to the positions of key points of the lips.
  • the extracting key points of lips from a portrait facial image includes:
  • the method before adjusting the positions of the key points of the lips according to the facial orientation and lip shape of the portrait, the method further includes:
  • the judging whether to adjust the position of the lip key points according to the shape of the lips and the corner key points of the lips key points includes:
  • the key point of the corner of the mouth does not meet the first preset condition, and the position of the key point of the lip is adjusted; wherein, the The first preset condition is that the brightness of the pixels in the middle of the mouth corner key points in the grayscale color space is lower than the first preset threshold;
  • the shape of the lips is an open mouth shape and the degree of mouth opening is greater than the second preset degree threshold
  • the key point of the corner of the mouth does not meet the second preset condition, the position of the key point of the lips is adjusted; wherein, the The first preset degree threshold is less than or equal to the second preset degree threshold; the second preset condition is that the mouth corner key point is located near the edge of the lip area and not located in the skin color area or lip color area at the same time.
  • the adjusting the position of the key points of the lips according to the face orientation and lip shape of the portrait includes:
  • the front face adjustment area expands a first preset number of pixels around according to the position of the key points of the side corners of the mouth the area formed by the points;
  • edge detection is performed on the front face adjustment area, and a first reference point is determined according to the first edge detection result, with the first reference point as the center , find the first skin color cut-off point and the first lip color cut-off point of the skin color area and the lip color area, and adjust the middle position between the first skin color cut-off point and the first lip color cut-off point as the key point of the side mouth corner s position;
  • the side deflection adjustment area is to expand the second preset number of pixels in the left and right direction according to the position of the key points of the side mouth corners and in the up and down direction expanding the area formed by the third preset number of pixels;
  • search for the darkest pixel in the side deviation adjustment area and perform edge detection on the side deviation adjustment area, according to the second Determine the second reference point based on the edge detection result, and search for the second skin color boundary point and the second lip color boundary point of the skin color area and the lip color area with the second reference point as the center, and adjust the side deviation according to the darkest pixel point
  • the relative position in the region, the second skin color cut-off point and the second lip color cut-off point adjust the positions of the key points of the side corners of the mouth.
  • the detection of the skin tone and skin color number of the face area in the portrait facial image includes:
  • the selecting a target lipstick color from a lipstick color sample library according to the skin tone and the skin color number includes:
  • each lipstick color interval According to the corresponding relationship between each lipstick color interval and the preset skin tone, find the target lipstick color interval corresponding to the skin tone of the human face area;
  • the corresponding lipstick color number is selected in the target lipstick color range.
  • an embodiment of the present disclosure provides an automatic makeup application device for portrait lips, including: a key point determination module, a color selection module, and a fusion coloring module; wherein,
  • the key point determination module is configured to extract lip key points from the portrait facial image, and detect the portrait face orientation and lip shape in the portrait facial image, and adjust the position of the lip key point according to the portrait facial orientation and lip shape;
  • the color selection module is configured to detect the skin tone and skin color number of the face area in the portrait facial image, and select the target lipstick color from the lipstick color sample library according to the skin tone and the skin color number;
  • the fusion coloring module is configured to use the target lipstick color to perform fusion coloring according to the positions of key points of the lips.
  • an embodiment of the present disclosure provides a computing device, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete mutual communication through the communication bus communication;
  • the memory is configured to store an executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned method for automatically applying makeup to portrait lips.
  • an embodiment of the present disclosure provides a computer storage medium, in which an executable instruction is stored, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned method for automatically applying makeup to portrait lips.
  • an embodiment of the present disclosure provides a computer program product, including computer readable code, and when the computer readable code is run in an electronic device, a processor in the electronic device executes the above-mentioned Operations corresponding to the automatic makeup application method for portrait lips.
  • the position of the lip key points after adjustment is ensured. Accuracy; and then automatically select the appropriate lipstick color and color it according to the skin tone and skin color number of the detected portrait facial image.
  • This solves the problem of inaccurate positions of key points and flaws in makeup when the light, angle, etc. Go to the step where the user chooses the lipstick color by himself, and automatically select the lipstick color and color it according to the corresponding conditions, which is more in line with human aesthetics, and improves the efficiency of makeup application and user experience.
  • Fig. 1a is a schematic flowchart of a method for automatically applying makeup to portrait lips provided by an embodiment of the present disclosure
  • Fig. 1b is a schematic diagram of the key points of a portrait facial image provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic flow diagram of adjusting the positions of key points of lips in the method for automatically applying makeup to portrait lips provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic flow diagram of automatic lipstick color selection in the method for automatically applying makeup to portrait lips provided by an embodiment of the present disclosure
  • FIG. 4 is a structural block diagram of an automatic makeup application device for portrait lips provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of a computing device provided by an embodiment of the present disclosure.
  • Fig. 1a is a schematic flowchart of a method for automatically applying makeup to portrait lips provided by an embodiment of the present disclosure. As shown in Fig. 1 , the method includes the following steps:
  • Step S101 extract lip key points from the portrait facial image, and detect the portrait facial orientation and lip shape in the portrait facial image.
  • the extraction of key points of lips from portrait facial images includes:
  • Fig. 1b is a schematic diagram of key points of a portrait facial image provided by an embodiment of the present disclosure. After recognizing the facial contour and five senses in the portrait facial image, the key points set for the portrait facial image are shown in Fig. 1b. Among them, Fig. 1b is a schematic diagram of the key points of the face image in the embodiment of the present disclosure. Next, the method for automatically applying makeup to the lips of a portrait in the embodiment of the present disclosure will be described by taking the key points of 68 standard settings in Fig. 1b as an example.
  • key points 49 to 68 are lip key points, which are used to describe the lip image information.
  • the detection of portrait facial orientation and lip shape in the portrait facial image includes:
  • the facial orientation includes: frontal orientation, left-biased orientation, and right-biased orientation; lip shape includes: mouth opened, mouth not opened;
  • the key point of the left lip is the lip key point on the left side of the vertical midline of the face, including: 49, 50, 51, 59, 60, 61, 62, 68;
  • the key point of the right lip is the right side of the vertical midline of the face Side lip key points, including: 53, 54, 55, 56, 57, 64, 65, 66.
  • the symmetrical left lip key points and right lip key points are the corresponding left lip key points and right lip key points, for example: left lip key point 50 and The key point 54 of the right lip is symmetrical with the midline of the face as the axis of symmetry, that is, the key point 50 of the left lip corresponds to the key point 54 of the right lip.
  • the upper lip key points are lip key points located above the horizontal midline of the lips except the left and right lip key points, including: 52 and 63 .
  • the lower lip key points are the lip key points located below the horizontal midline of the lips except the left and right lip key points, including: 58 and 67 .
  • the key points of the upper lip and the lower lip that are symmetrical up and down are the corresponding upper lip key points and lower lip key points, for example, the upper lip key point 52 and the lower lip key point
  • the lower lip keypoint 58 corresponds.
  • calculating the first position difference requires extracting any two key points from the left lip key points, and the position difference between the extracted two key points in the x-axis direction is the first position difference . For example, extract key points 49 and 50 of lips in the key points of lips on the left side, then the first position difference is the position difference between key points 49 and 50 of lips in the x-axis direction;
  • Calculating the second position difference needs to extract two key points corresponding to the two extracted left key points from the right lip key point, and the extracted two key points have The position difference is the second position difference. For example, if the left lip key points 49 and 50 are extracted, then the corresponding lip key points 55 and 54 are extracted from the right lip key points, then the second position difference is the lip key points 55 and 54 in the x-axis direction position difference on
  • Calculating the third position difference requires extracting an upper lip key point and a corresponding lower lip key point, and the position difference between the extracted two key points in the y-axis direction is the third position difference .
  • extract 63 from the upper lip key point then extract 67 from the lower lip key point
  • the third position difference is the position difference between the lip key points 63 and 67 in the y-axis direction.
  • the width, height, image resolution, the first position difference and the second position difference of the face area in the portrait face image determine the face orientation of the portrait; wherein, the face orientation of the portrait includes: front face orientation, left Offset and rightward orientation;
  • the shape of the lips is determined according to the width, height, image resolution and the third position difference of the face area in the face image of the portrait.
  • the method for judging the face orientation may be:
  • d1 is the position difference between key points 51 and 50 in the x-axis direction, that is, the first position difference
  • d2 is the position difference between key points 54 and 53 in the x-axis direction, that is, the second position difference
  • R w is the width of the human face area
  • Rh is the height of the human face area
  • WH is the image resolution, P52 .
  • the method of judging the shape of the lips can be:
  • open y is the position difference of key points 67 and 63 in the y-axis direction, that is, the third position difference;
  • P58.y- P67.y is the position difference of key points 67 and 58 in the y-axis direction; in addition , set the first preset degree threshold Y 1 , the second preset degree threshold Y 2 , and Y 2 >Y 1 .
  • the lip shape When the lip shape is open mouth, if open y ⁇ Y 1 , it is judged that the lip shape is a small mouth opening degree, that is, the mouth opening degree is less than the first preset degree threshold; if open y >Y 2 , it is judged that the lip shape is a lower mouth opening degree Large, that is, the degree of mouth opening is greater than the second preset degree threshold.
  • Step S102 adjusting the positions of the key points of the lips according to the face orientation and lip shape of the portrait.
  • the first preset condition is that, in the grayscale color space, the brightness of the pixels in the middle of the mouth key points is lower than the first preset threshold;
  • the shape of the lips is an open mouth shape and the degree of mouth opening is greater than the second preset degree threshold
  • the key point of the corner of the mouth does not meet the second preset condition, the position of the key point of the lips is adjusted; wherein, the The first preset degree threshold is less than or equal to the second preset degree threshold; the second preset condition is that the mouth corner key point is located near the edge of the lip area and is not located in the skin color area or lip color area at the same time.
  • the middle point of the key points of the two corners of the mouth should be at the junction of the upper and lower lips. Compared with the surrounding areas, the middle point The shaded area in the depression. Therefore, in the grayscale color space, if the brightness of the point is lower than the brightness of the surrounding macroblocks, it means that it is in the shadow area, that is, it can be judged that the orientation of the portrait has not changed or the shape of the lips has not changed. Conversely, if the orientation of the portrait changes or the shape of the lips changes, the center point is not in the shadow area, so in the grayscale color space, the brightness of the center point is not necessarily lower than the brightness of the surrounding macroblocks.
  • the first preset condition may be: in the grayscale color space, the brightness of the pixel points in the middle of the mouth key point is lower than the first preset threshold. Then when the shape of the lips is not open or the degree of mouth opening is less than the first preset threshold, if in the grayscale color space, the brightness of the pixel in the middle of the mouth key point is not lower than the first preset threshold, Then adjust the position of the key points of the lips;
  • the first preset threshold may be 10% of the darkest brightness in the lip area.
  • the key points of the two corners of the mouth are near the edges of the surrounding macroblocks, and at the same time, the key points of the corners of the mouth are only located in the skin color area Or the lip color area; on the contrary, if the orientation of the portrait changes or the shape of the lips changes, the position of the key point of the mouth corner also changes, and it will no longer be at the edge of the surrounding macroblock, and the skin color area and lip color area where it is located If the color areas may overlap, you need to adjust the position of the key points of the corner of the mouth.
  • the second preset condition may be: the mouth corner key point is located near the edge of the lip area and is not located in the skin color area or the lip color area at the same time. Then when the shape of the lips is an open mouth shape and the degree of mouth opening is greater than the second preset degree threshold, if the key point of the mouth corner is located near the edge of the lip area and is not located in the skin color area or lip color area at the same time, the lip The position of the key point is adjusted; wherein, the first preset degree threshold is less than or equal to the second preset degree threshold.
  • Step S103 detecting the skin tone and skin color number of the face area in the portrait facial image.
  • the corresponding skin tone judgment coefficient is calculated, and the skin tone of the face area in the portrait face image is judged according to the skin tone judgment coefficient.
  • Hue Calculate the corresponding skin color number judgment coefficient according to the color parameters of the face area in the portrait face image; judge the skin color number of the face area according to the skin color number judgment coefficient.
  • Step S104 selecting a target lipstick color from a lipstick color sample library according to the skin tone and the skin color number.
  • all lipstick colors in the lipstick color sample library are divided into intervals according to preset color matching coefficients, and the corresponding relationship between each lipstick color interval and preset skin tone is configured;
  • each lipstick color interval According to the corresponding relationship between each lipstick color interval and the preset skin tone, find the target lipstick color interval corresponding to the skin tone of the human face area;
  • the corresponding lipstick color number is selected in the target lipstick color range.
  • the preset color matching coefficient can be:
  • ⁇ L is the average value of the brightness L of the human face area in the Lab color space
  • ⁇ stone is the skin tone judgment coefficient of the human face area
  • ⁇ level is the skin color number judgment coefficient of the human face area
  • L max is the human face area in the Lab color space The maximum value of brightness L
  • L min is the minimum value of brightness L of the face area in the Lab color space.
  • the lipstick colors in the lipstick color sample library are divided into intervals according to the color matching coefficient ⁇ , and the lipstick color intervals from blue tone to yellow tone are divided.
  • Step S105 performing fusion coloring with the target lipstick color according to the positions of key points of the lips.
  • the lip image information in the facial image of the portrait is accurately described, and based on this, the lips of the portrait are fused and colored; wherein, the color used for fused and colored is a step S104 determines the lipstick color corresponding to the lipstick number.
  • key points of the lips are extracted from the face image of the portrait, and the orientation of the face of the portrait and the shape of the lips are detected, and the positions of the key points of the lips are adjusted.
  • detect the skin tone and skin color number of the face area in the portrait facial image and select the target lipstick color from the lipstick color sample library.
  • the lips of the portrait are blended and colored according to the positions of the key points of the lips and the selected lipstick color.
  • the face orientation and lip shape of the portrait can be detected, and the position of the key points of the lips in the portrait can be adjusted, and the correction caused by different facial orientation and lip shape can be corrected.
  • the key point position changes brought about by the shape, so that the adjusted key point position conforms to the face orientation and lip shape of the current portrait; and based on the detected skin tone and skin color number, it automatically selects a suitable lipstick number for the portrait lips. Fusion coloring.
  • Fig. 2 is a schematic flow chart of adjusting the position of the key points of the lips in the automatic makeup method for portrait lips provided by the embodiment of the present disclosure. As shown in Fig. 2, the method includes the following steps:
  • Step S201 respectively determine the lip color judgment criterion of the lip color area and the skin color judgment criterion of the skin color area.
  • the mean variance ratio of a-b in the Lab color space is calculated as the lip color judgment criterion in the lip color area; any area is found in the cheek region, and the mean variance ratio of a-b in the Lab color space is calculated as the skin color judgment criterion.
  • the lip color area is determined according to key points 57, 58, 59, 66, 67, 68.
  • Step S202 detecting the face orientation and lip shape of the face in the face image of the face.
  • step S101 of the foregoing embodiment The manner of detecting the face orientation and lip shape of the face in the face image of the face has been described in detail in step S101 of the foregoing embodiment.
  • face orientation is divided into frontal orientation, left orientation, and right orientation; lip shape is divided into: mouth open, mouth not open.
  • Step S203 selecting a corresponding adjustment method to adjust the position of key points of the lips according to the orientation of the face of the face and the shape of the lips in the face image of the face.
  • the facial orientation of the portrait is the frontal orientation and the lip shape is the unopened mouth shape
  • search for the darkest two pixel points in one side of the frontal face adjustment area and use the middle position of the two pixel points as the The adjusted position of the side mouth corner key points;
  • the front face adjustment area is an area formed by extending a first preset number of pixels around according to the position of the side mouth corner key points. Since the portrait is facing the front face and does not open the mouth, the darkest part of the adjustment area on one side of the front face is the corner of the mouth that is in the shadow due to the depression, and the middle position of the two darkest pixels can be more accurately determined.
  • the position of the corner of the mouth on the side; at the same time, according to the position of the key point of the corner of the mouth on the side to expand the first preset number of pixels to determine the front face adjustment area can determine the approximate position of the region based on the position of the key point of the corner of the mouth, and the extended range can Ensuring sufficient room for adjustment while maintaining accuracy.
  • the first preset number may be 10, that is, the area formed by extending 10 pixels around the key point of the side mouth corner is the frontal face adjustment area.
  • edge detection is performed on the front face adjustment area, and a first reference point is determined according to the first edge detection result, with the first reference point as the center , find the first skin color cut-off point and the first lip color cut-off point of the skin color area and the lip color area, and adjust the middle position between the first skin color cut-off point and the first lip color cut-off point as the key point of the side mouth corner s position.
  • the first reference point of the original mouth corner area is determined by edge detection, and since the mouth corner position is the boundary between skin color and lip color, the first skin color boundary point and the first lip color boundary point are determined based on the first reference point.
  • the middle position of the former can accurately represent the adjusted position of the key points of the side corners of the mouth.
  • the first skin color cut-off point is determined according to the skin color judgment criterion of the skin color area, and the first lip color cut-off point is determined according to the lip color judgment criterion;
  • the key points of the side corner of the mouth include: the key point of the left corner of the mouth and the key point of the right corner of the mouth, wherein the key point of the left corner of the mouth can include key points 49 and 61, and the key point of the right corner of the mouth can include key points 55 and 65.
  • the adjustment of the key points of the side mouth corner includes the adjustment of the key point of the left mouth corner and the adjustment of the key point of the right mouth corner.
  • the edge detection of the frontal face adjustment area can be Laplace edge detection of grayscale images on the frontal face adjustment area; Among the edge points in the result, select the point closest to No. 49 and No. 61 key points as the first reference point; in the same way, for the adjustment of the key points of the corner of the mouth on the right side, it can be selected from the edge points in the first edge detection result The point closest to key points 55 and 65 is used as the first reference point.
  • the facial orientation of the portrait is left-biased or right-biased and the shape of the lips is not open, search for the darkest pixel in the side deviation adjustment area, and adjust the side deviation according to the darkest pixel.
  • the relative position in the area is to adjust the position of the key points of the side corners of the mouth; wherein, the side deflection adjustment area is to expand the second preset number of pixels in the left and right direction according to the position of the key points of the side mouth corners and in the up and down direction
  • the area formed by the third preset number of pixels is expanded. Due to the change of the facial orientation of the portrait, each key point also moves when it is facing left. By finding the darkest point and determining the positional relationship between the darkest point and the key point of the original corner of the mouth, the position of the key point can be determined more accurately.
  • the side deviation adjustment area can be divided into a left mouth corner adjustment area and a right mouth corner adjustment area, the left mouth corner adjustment area is determined according to the left mouth corner key points, and the right mouth corner adjustment area is determined according to the right mouth corner key points.
  • the adjustment area of the left mouth corner is to extend P61. x -P49. x pixels to the left with key point 49, to expand P61 .
  • the area formed by extending the point by 10 pixels up and down; the adjustment area of the right mouth corner is to extend P65. x -P55. x pixels to the right with key point 55, and to expand P65. x -P55 . pixels, and expand the area formed by 10 pixels up and down.
  • search for the darkest pixel in the side deviation adjustment area and perform edge detection on the side deviation adjustment area, according to the second Determine the second reference point based on the edge detection result, and search for the second skin color boundary point and the second lip color boundary point of the skin color area and the lip color area with the second reference point as the center, and adjust the side deviation according to the darkest pixel point
  • the relative position in the region, the second skin color cut-off point and the second lip color cut-off point adjust the positions of the key points of the side corners of the mouth.
  • the darkest point on the edge is determined by edge detection, and then based on the positional relationship between the darkest point and the key point of the original mouth corner, the darkest point and The skin color cut-off point or lip color cut-off point can accurately determine the key points of the adjusted mouth corners.
  • the second skin color cut-off point is determined according to the skin color judgment criterion of the skin color area, and the second lip color cut-off point is determined according to the lip color judgment criterion;
  • performing edge detection on the side deviation adjustment area may perform Laplace edge detection on grayscale images on the side deviation adjustment area; Among the edge points in , the point closest to No. 49 and No. 61 key points is used as the second reference point; similarly, for the adjustment of the key points of the corner of the mouth on the right side, the distance 55 can be selected from the edge points in the second edge detection result. The point closest to key point 65 is used as the second reference point.
  • the side deviation adjustment area can be divided into a left mouth corner adjustment area and a right mouth corner adjustment area, and the left mouth corner adjustment area is determined according to the key points of the left mouth corner.
  • Side corners of the mouth adjustment area determine the right side corners of the mouth adjustment area according to the key points of the right side mouth corners;
  • the left side mouth corners adjustment area is to expand P61 .
  • Expand P61. x -P49. x pixels to the right and expand the area formed by 10 pixels up and down with No. 49 and No. 61 key points;
  • the mouth corner adjustment area on the right is to expand P65 . -P55. x pixels, expand P65. x -P55. x pixels to the left with No. 65 key point, expand the area formed by 10 pixels up and down.
  • the darkest pixel in the left mouth corner adjustment area is between key points 49 and 61, use the middle position between key points 49 and 61 as the key point adjustment for the left mouth corner
  • the darkest pixel in the left mouth corner adjustment area is on the left side of No. 49 key point, then extend the darkest point to the left to the second skin color boundary point, and use the darkest point and the No.
  • the middle position of the second skin color critical point is used as the adjusted position of the key point of the left corner of the mouth; if the darkest pixel point in the adjustment area of the left corner of the mouth is on the right side of key point 61, the darkest point is extended to the right to the No. 61 key point
  • the cutoff point of the second lip color, the adjusted position of the key point of the left corner of the mouth is taken as the middle position between the darkest point and the second lip color critical point.
  • the darkest pixel in the right mouth corner adjustment area is between key points 55 and 65, then take the middle position of key points 55 and 65 as the adjusted position of the right mouth corner key points; if the right mouth corner adjustment area If the darkest pixel is on the right side of key point 55, extend the darkest point to the right to the second skin color cut-off point, and use the middle position between the darkest point and the second skin color cut-off point as the right corner of the mouth
  • the middle position between the point and the second lip color critical point is used as the adjusted position of the key point of the right corner of the mouth.
  • FIG. 3 is a schematic flow chart of automatically selecting a lipstick color in the method for automatically applying makeup to the lips of a portrait provided by an embodiment of the present disclosure. As shown in FIG. 3 , the method includes the following steps:
  • Step S301 determining the skin tone of the face area in the face image of the portrait.
  • the skin tone of the human face region in the portrait facial image includes: cool tone, warm tone, and neutral tone; according to the color parameters of the human face region in the portrait facial image, the corresponding skin tone judgment coefficient; when the skin tone judgment coefficient is within the first preset interval, it is determined that the skin tone is a cool tone; when the skin tone judgment coefficient is within the second preset interval, then Determining that the skin tone is a warm tone; when the skin tone judgment coefficient is not within the first preset range and not within a second preset range, then it is determined that the skin tone is a neutral tone.
  • the portrait face image is transformed from RGB color space to HSI color space; wherein, H is hue (Hue), S is saturation (Saturation or Chroma), and I is brightness (Intensity or Brightness).
  • hue H saturation S and brightness I of the face area in the portrait face image, calculate the mean value, variance, maximum value, and minimum value of the hue H; calculate the mean value and variance of the saturation S.
  • the mean value of b of the eye area is calculated; where, b is the position coordinate between yellow and blue in the Lab color space; if b When >0, it means that the color of the area is close to yellow, and the skin is warm.
  • the calculation method of the skin tone judgment coefficient is:
  • H is the hue of the face area in the HSI color area.
  • H is reassigned to H-2 ⁇ ; in other cases, the value of H is the original value of H.
  • ⁇ H is the mean value of the hue H of the face area
  • ⁇ S is the mean value of the saturation S of the face area
  • ⁇ b is the mean value of the b coordinate of the eyeball area
  • ⁇ H is the variance of the hue H of the face area
  • ⁇ S is the face area The variance of the saturation S
  • H max is the maximum value of the hue H of the face area
  • H min is the minimum value of the hue H of the face area.
  • Step S302 determining the skin color number of the face area in the portrait facial image.
  • the corresponding skin color number judgment coefficient is calculated; and the skin color number of the face area is judged according to the skin color number judgment coefficient.
  • calculating the HSI color space calculate the mean value and variance of the hue H; calculate the mean value, variance, maximum value, and minimum value of the brightness I;
  • the calculation method of the skin color number judgment coefficient is:
  • H is the hue of the face area in the HSI color area.
  • H is reassigned to H-2 ⁇ ; in other cases, the value of H is the original value of H.
  • ⁇ I is the mean value of the brightness I of the face area
  • ⁇ H is the mean value of the tone H of the face area
  • ⁇ H is the variance of the tone H of the face area
  • ⁇ I is the variance of the brightness I of the face area
  • I min is the minimum value of brightness I of the face area.
  • the preset skin color number may include: a first color number, a second color number and a third color number;
  • ⁇ level ⁇ TH 0 determines that the skin color number of the human face area is the first color number; when ⁇ level >TH 1 , then determine that the skin color number of the human face area is the third color number; In other cases, it is determined that the skin color number of the human face area is the second color number; wherein, TH 0 and TH 1 are preset color number thresholds for distinguishing color numbers.
  • Step S303 divide all the lipstick colors in the lipstick color sample database into intervals, and configure the corresponding relationship between each lipstick color interval and the preset skin tone.
  • all lipstick colors in the lipstick color sample library are divided into intervals according to preset color matching coefficients, and then the corresponding relationship between each lipstick color interval and preset skin tone is configured; wherein, the preset skin tone Includes: Cool, Warm, Neutral.
  • the preset color matching coefficient of the color matching coefficient can be any color matching coefficient.
  • ⁇ stone is the skin tone judgment coefficient of the human face area
  • ⁇ level is the skin color number judgment coefficient of the human face area
  • ⁇ L is the mean value of the brightness L of the human face area in the Lab color space
  • L max is the human face area in the Lab color space The maximum value of brightness L
  • L min is the minimum value of brightness L of the face area in the Lab color space.
  • the correspondence between each lipstick color interval and the preset skin tone can be:
  • Cool tones correspond to the lipstick color range of the blue-green color range
  • warm colors correspond to the lipstick color range of the red-yellow color range
  • neutral tones can correspond to the lipstick color range of all color systems.
  • Step S304 according to the correspondence between each lipstick color interval and the preset skin tone, search for the target lipstick color interval corresponding to the skin tone of the face area.
  • the portrait is found according to the skin tone of the face area in the portrait facial image determined in step S301, and the correspondence between each lipstick color range and preset skin tone determined in step S304 The lipstick color range corresponding to the face image.
  • Step S305 according to the skin color number of the face area, select the corresponding lipstick color number in the target lipstick color interval.
  • the lipstick color interval found in step S304 is selected to suit the yellowish-white skin color of the portrait facial image in some implementations. lipstick number.
  • FIG. 4 is a structural block diagram of an automatic makeup application device for portrait lips provided by an embodiment of the present disclosure. As shown in FIG. 4 , the device includes: a key point determination module 401, a color selection module 402, and a fusion coloring module 403; wherein,
  • the key point determination module 401 is configured to extract lip key points from the portrait facial image, and detect the portrait face orientation and lip shape in the portrait facial image, and adjust the position of the lip key point according to the portrait facial orientation and lip shape .
  • the key point determination module 401 is further configured to: identify the facial contour and facial features in the facial image of the portrait, set key points for the facial image of the portrait, and extract the key points of lips from all the key points; wherein, The lip key points are used to describe the lip image information.
  • the key point determination module 401 is further configured to: judge whether to adjust the position of the lip key points according to the shape of the lips and the corner key points of the lip key points.
  • the key point determination module 401 is further configured to: when the shape of the lips is not open or the degree of mouth opening is less than a first preset degree threshold, if the key point of the corner of the mouth does not meet the first preset If a condition is set, the position of the key point of the lips is adjusted; wherein, the first preset condition is that the brightness of the pixel in the middle of the key point of the corner of the mouth in the grayscale color space is lower than the first preset threshold;
  • the shape of the lips is an open mouth shape and the degree of mouth opening is greater than the second preset degree threshold
  • the key point of the corner of the mouth does not meet the second preset condition, the position of the key point of the lips is adjusted; wherein, the The first preset degree threshold is less than or equal to the second preset degree threshold; the second preset condition is that the mouth corner key point is located near the edge of the lip area and not located in the skin color area or lip color area at the same time.
  • the key point determining module 401 is further configured to: for the key point of the side mouth corner in the key points of the lips, when the face orientation of the portrait is the front face orientation and the lip shape is not opened In the form, search for the darkest two pixels in the front face adjustment area, and use the middle position of the two pixels as the adjusted position of the key points of the side corners of the mouth; wherein, the front face adjustment area is based on the The position of the key point of the side mouth corner is extended to the surrounding area formed by the first preset number of pixels;
  • edge detection is performed on the front face adjustment area, and a first reference point is determined according to the first edge detection result, with the first reference point as the center , find the first skin color cut-off point and the first lip color cut-off point of the skin color area and the lip color area, and adjust the middle position between the first skin color cut-off point and the first lip color cut-off point as the key point of the side mouth corner s position;
  • the side deflection adjustment area is to expand the second preset number of pixels in the left and right direction according to the position of the key points of the side mouth corners and in the up and down direction expanding the area formed by the third preset number of pixels;
  • search for the darkest pixel in the side deviation adjustment area and perform edge detection on the side deviation adjustment area, according to the second Determine the second reference point based on the edge detection result, and search for the second skin color boundary point and the second lip color boundary point of the skin color area and the lip color area with the second reference point as the center, and adjust the side deviation according to the darkest pixel point
  • the relative position in the region, the second skin color cut-off point and the second lip color cut-off point adjust the positions of the key points of the side corners of the mouth.
  • the color selection module 402 is configured to detect the skin tone and skin color number of the face area in the portrait facial image, and select the target lipstick color from the lipstick color sample library according to the skin tone and the skin color number .
  • the color selection module 402 is further configured to: calculate the corresponding skin tone judgment coefficient according to the color parameters of the face area in the portrait facial image; when the skin tone judgment coefficient is in the first When it is within a preset interval, it is determined that the skin tone is a cool tone; when the skin tone judgment coefficient is within a second preset interval, it is determined that the skin tone is a warm tone; when the skin tone When the judgment coefficient is not in the first preset interval and not in the second preset interval, it is determined that the skin tone is a neutral tone; according to the color parameters of the face area in the portrait facial image, the corresponding The skin color number judgment coefficient of the skin color number; judge the skin color number of the human face area according to the skin color number judgment coefficient.
  • the color selection module 402 is further configured to: divide all lipstick colors in the lipstick color sample library into intervals according to preset color matching coefficients, and configure the distance between each lipstick color interval and the preset skin tone Correspondence: According to the correspondence between each lipstick color interval and the preset skin tone, find the target lipstick color interval corresponding to the skin tone of the human face area; according to the skin color number of the human face area, in the target lipstick color Select the corresponding lipstick number in the interval.
  • the fusion coloring module 403 is configured to use the target lipstick color to perform fusion coloring according to the positions of key points of the lips.
  • the fusion coloring module 403 is further configured to: according to the positions of the key points of the lips determined by the key point determination module 401 and the corresponding lipstick number selected by the color selection module 402, for the face image of the portrait Color the lip area.
  • key points of lips are extracted from a facial image of a portrait, and the orientation of the face of a portrait and the shape of lips are detected to adjust the position of the key points of lips.
  • detect the skin tone and skin color number of the face area in the portrait facial image and select the target lipstick color from the lipstick color sample library.
  • the lips of the portrait are blended and colored according to the positions of the key points of the lips and the selected lipstick color.
  • the embodiments of the present disclosure it is possible to detect the face orientation and lip shape of a portrait based on the key points of lips extracted from the face image of the portrait, and adjust the position of the key points of the lips in the portrait, correcting the key points caused by different face orientations and lips.
  • the key point position changes brought about by the shape, so that the adjusted key point position conforms to the face orientation and lip shape of the current portrait; and based on the detected skin tone and skin color number, it automatically selects a suitable lipstick number for the portrait lips. Fusion coloring.
  • An embodiment of the present disclosure also provides a non-volatile computer storage medium, where an executable instruction is stored in the computer storage medium, and the executable instruction can execute the method for automatically applying makeup to portrait lips in any of the above method embodiments.
  • FIG. 5 is a schematic structural diagram of a computing device provided by an embodiment of the present disclosure, and the embodiment of the present disclosure does not limit the implementation of the computing device.
  • the computing device may include: a processor (processor) 502, a communication interface (Communications Interface) 504, a memory (memory) 506, and a communication bus 508.
  • processor processor
  • Communication interface Communication Interface
  • memory memory
  • the processor 502 , the communication interface 504 , and the memory 506 communicate with each other through the communication bus 508 .
  • the communication interface 504 is configured to communicate with network elements of other devices such as clients or other servers.
  • the processor 502 is configured to execute the program 510, and may execute the relevant steps in the above embodiment of the method for automatically applying makeup to the lips of a portrait.
  • the program 510 may include program code including computer operating instructions.
  • the processor 502 may be a central processing unit CPU, or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present disclosure.
  • the one or more processors included in the computing device may be of the same type, such as one or more CPUs, or may be different types of processors, such as one or more CPUs and one or more ASICs.
  • the memory 506 is configured to store the program 510 .
  • the memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as a disk memory.
  • the program 510 is configured to enable the processor 502 to execute the method for automatically applying makeup to the lips of a portrait in any of the above method embodiments.
  • each step in the program 510 refer to the corresponding steps and the corresponding descriptions in the units in the above-mentioned embodiment of the method for automatically applying makeup to the lips of a portrait.
  • the working process of the above-described devices and modules can refer to the corresponding process description in the foregoing method embodiments.
  • An embodiment of the present disclosure provides a computer program product, including computer readable code.
  • a processor in the electronic device executes any of the above method embodiments.
  • modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment.
  • Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies.
  • All features disclosed in this specification including accompanying claims, abstract and drawings) and any method or apparatus so disclosed may be used in any combination, except that some of such features and/or processes or units are mutually exclusive. All processes or units are combined.
  • Each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
  • the various component embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components according to the present disclosure.
  • DSP digital signal processor
  • the present disclosure can also be implemented as an apparatus or apparatus program (eg, computer program and computer program product) for performing a part or all of the methods described herein.
  • Such a program realizing the present disclosure may be stored on a computer-readable medium, or may have the form of one or more signals.
  • Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.
  • the accuracy of the position of the key points of the lips after adjustment is ensured; and then according to the detected portrait
  • the skin tone and skin color number of the face image automatically selects the appropriate lipstick color and paints it. This solves the problem of inaccurate positions of key points and flaws in makeup when the light, angle, etc. Go to the step where the user chooses the lipstick color by himself, and automatically select the lipstick color and color it according to the corresponding conditions, which is more in line with human aesthetics, and improves the efficiency of makeup application and user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例提供了一种人像唇部的自动上妆方法、装置、设备、存储介质及程序产品，其中，该方法包括：从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置；检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色；依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。通过本公开实施例可以更为准确地确定人像中的嘴角关键点，使自动上妆位置更为准确也更符合人类审美，同时，通过判断人像面部图像中的人脸区域的皮肤色调和皮肤色号，自动选择适合的口红色号进行上色，不再需要用户手动上色。

Description

人像唇部的自动上妆方法、装置、设备、存储介质及程序产品
相关申请的交叉引用
本公开基于申请号为202110982907.2、申请日为2021年08月25日、申请名称为“一种人像唇部的自动上妆方法、装置、设备和存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以全文引入的方式引入本公开。
技术领域
本公开涉及音频、视频及图像处理领域，尤其涉及一种人像唇部的自动上妆方法、装置、设备、存储介质及程序产品。
背景技术
随着智能终端图像处理技术的发展，智能终端在通过相机采集人像信息后，可以按照用户选择对所采集人像进行自动上妆，这项功能已经广泛应用于智能终端自身的照相、摄像应用，以及互联网直播时对实时采集的人像的处理。其中，对嘴唇上妆时，先通过人像对嘴唇轮廓进行确定，再依据用户选择的口红色号进行上色。
而在摄像或互联网直播过程中，人像尤其是面部区域几乎时刻处于动态变化中。而相关技术在嘴唇上妆时，按照人脸检测关键点标注的基础上进行曲线拟合并上色，对于关键点的位置不做调整，导致当光线或人脸角度发生变化时，嘴唇及嘴角的关键点位置不准，与实际位置不符，对上妆效果产生较大影响。
发明内容
鉴于上述问题，提出了本公开实施例的人像唇部的自动上妆方法、装置，设备、存储介质及程序产品。
第一方面，本公开实施例提供了一种人像唇部的自动上妆方法，所述方法包括：
从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置；
检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色；
依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
在一种可能的实现方式中，所述从人像面部图像中提取嘴唇关键点，包括：
识别人像面部图像中的面部轮廓及五官，为人像面部图像设置关键点，从全部关键点中提取嘴唇关键点；其中，所述嘴唇关键点用于描述嘴唇图像信息。
在一种可能的实现方式中，在所述根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置之前，所述方法还包括：
根据嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整。
在一种可能的实现方式中，所述根据嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整，包括：
当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，所述嘴角关键点不符合第一预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设条件为在灰度色彩空间中所述嘴角关键点中间的像素点的亮度低于第一预设阈值；
当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点不符合第二预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值；所述第二预设条件为所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。
在一种可能的实现方式中，所述根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置，包括：
针对所述嘴唇关键点中的侧边嘴角关键点，当所述人像面部朝向为正脸朝向且所述嘴唇形态为未张嘴形态时，在正脸调整区域中查找最暗的两个像素点，将所述两个像素点的中间位置作为侧边 嘴角关键点调整后的位置；其中，所述正脸调整区域是根据所述侧边嘴角关键点的位置向四周扩展第一预设数量的像素点所形成的区域；
当所述人像面部朝向为正脸朝向且所述嘴唇形态为张嘴形态时，对正脸调整区域进行边缘检测，依据第一边缘检测结果确定第一参照点，以所述第一参照点为中心，查找肤色区域和唇色区域的第一肤色分界点和第一唇色分界点，将所述第一肤色分界点和所述第一唇色分界点的中间位置作为侧边嘴角关键点调整后的位置；
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为未张嘴形态时，在侧偏调整区域中查找最暗的像素点，依据所述最暗的像素点在侧偏调整区域中的相对位置，调整侧边嘴角关键点的位置；其中，所述侧偏调整区域是根据所述侧边嘴角关键点的位置向左右方向扩展第二预设数量的像素点且向上下方向扩展第三预设数量的像素点所形成的区域；
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为张嘴形态时，在侧偏调整区域中查找最暗的像素点，并对侧偏调整区域进行边缘检测，依据第二边缘检测结果确定第二参照点，以所述第二参照点为中心，查找肤色区域和唇色区域的第二肤色分界点和第二唇色分界点，依据最暗的像素点在侧偏调整区域中的相对位置、所述第二肤色分界点以及所述第二唇色分界点，调整侧边嘴角关键点的位置。
在一种可能的实现方式中，所述检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，包括：
根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数；当所述皮肤色调判断系数处于第一预设区间之内时，则判定所述皮肤色调为冷色调；当所述皮肤色调判断系数处于第二预设区间之内时，则判定所述皮肤色调为暖色调；当所述皮肤色调判断系数不处于第一预设区间且不处于第二预设区间之内时，则判定所述皮肤色调为中性色调；
根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
在一种可能的实现方式中，所述根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色，包括：
根据预设的配色系数对口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系；
依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找所述人脸区域的皮肤色调对应的目标口红颜色区间；
根据所述人脸区域的皮肤色号，在目标口红颜色区间中选择对应的口红色号。
第二方面，本公开实施例提供了一种人像唇部的自动上妆装置，包括：关键点确定模块、颜色选择模块以及融合上色模块；其中，
所述关键点确定模块，配置为从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置；
所述颜色选择模块，配置为检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色；
所述融合上色模块，配置为依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
第三方面，本公开实施例提供了一种计算设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
所述存储器配置为存放一可执行指令，所述可执行指令使所述处理器执行如上述的人像唇部的自动上妆方法对应的操作。
第四方面，本公开实施例提供了一种计算机存储介质，存储介质中存储有一可执行指令，可执行指令使处理器执行如上述的人像唇部的自动上妆方法对应的操作。
第五方面，本公开实施例提供了一种计算机程序产品，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行如上述的人像唇部的自动上妆方法对应的操作。
根据本公开实施例提供的技术方案，通过从人像面部图像提取嘴唇关键点，并根据检测到的人像面部朝向及嘴唇形态，调整所述嘴唇关键点的位置，确保了调整后嘴唇关键点位置的准确性；然后再根据检测到的人像面部图像的皮肤色调和皮肤色号自动选择合适的口红颜色并上色。由此解决了相关技术中按照人脸检测关键点标注的基础上进行曲线拟合并上色，在人像光线、角度等发生变化时关键点位置不准确、上妆有瑕疵的问题，同时，省去了用户自己选择口红颜色的步骤，根据相 应条件自动选择口红颜色并上色，更符合人类审美，提高了上妆的效率以及用户体验。
上述说明是本公开实施例技术方案的概述，为了能够更清楚了解本公开实施例的技术手段，而可依照说明书的内容予以实施，并且为了让本公开实施例的上述和其它目的、特征和优点能够更明显易懂，以下特举本公开实施例的具体实施方式。
附图说明
通过阅读下文各实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图用于示出各实施方式的目的，而并不认为是对本公开实施例的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1a为本公开实施例提供的人像唇部的自动上妆方法的流程示意图；
图1b为本公开实施例提供的人像面部图像关键点的示意图；
图2为本公开实施例提供的人像唇部的自动上妆方法中调整嘴唇关键点位置的流程示意图；
图3为本公开实施例提供的人像唇部的自动上妆方法中自动选择口红颜色的流程示意图；
图4为本公开实施例提供的人像唇部的自动上妆装置的结构框图；
图5为本公开实施例提供的一种计算设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
图1a为本公开实施例提供的人像唇部的自动上妆方法的流程示意图，如图1所示，该方法包括如下步骤：
步骤S101，从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态。
在一些实施方式中，所述从人像面部图像中提取嘴唇关键点，包括：
识别人像面部图像中的面部轮廓及五官，为人像面部图像设置关键点，从全部关键点中提取嘴唇关键点；其中，所述嘴唇关键点用于描述嘴唇图像信息。
图1b为本公开实施例提供的人像面部图像关键点的示意图，在识别人像面部图像中的面部轮廓及五观后，为人像面部图像设置的关键点如图1b所示。其中，图1b为本公开实施例中面部图像关键点示意图，下面，以图1b中的68点标准设置的关键点为例来说明本公开实施例中的人像唇部的自动上妆方法。
图1b中所示的全部关键点中，以49、55、51、53、58为边界，向外扩展20个像素作为嘴唇区域，49至68号关键点为嘴唇关键点，用于描述嘴唇图像信息。
在一些实施方式中，所述检测人像面部图像中的人像面部朝向以及嘴唇形态，包括：
所述面部朝向包括：正脸朝向、左偏朝向、右偏朝向；嘴唇形态包括：张嘴、未张嘴；
从所述嘴唇关键点49至68中提取左侧嘴唇关键点、与所述左侧嘴唇关键点对应的右侧嘴唇关键点、上侧嘴唇关键点以及与所述上侧嘴唇关键点对应的下侧嘴唇关键点；
其中，所述左侧嘴唇关键点为面部垂直中线左侧的嘴唇关键点，包括：49、50、51、59、60、61、62、68；所述右侧嘴唇关键点为面部垂直中线右侧的嘴唇关键点，包括：53、54、55、56、57、64、65、66。另外，以面部垂直中线为对称轴，相互对称的左侧嘴唇关键点和右侧嘴唇关键点，即为对应的左侧嘴唇关键点和右侧嘴唇关键点，例如：左侧嘴唇关键点50与右侧嘴唇关键点54以面部中线为对称轴相对称，即左侧嘴唇关键点50与右侧嘴唇关键点54相对应。所述上侧嘴唇关键点为除左侧和右侧嘴唇关键点外，位于嘴唇水平中线上侧的嘴唇关键点，包括：52和63。所述下侧嘴唇关键点为为除左侧和右侧嘴唇关键点外，位于嘴唇水平中线下侧的嘴唇关键点，包括：58和67。另外，以嘴唇水平中线为对称轴，上下对称的上侧嘴唇关键点与下侧嘴唇关键点，即为相互对应的上侧嘴唇关键点和下侧嘴唇关键点，例如上侧嘴唇关键点52与下侧嘴唇关键点58相对应。
计算所述左侧嘴唇关键点在x轴方向上的第一位置差、所述右侧嘴唇关键点在x轴方向上的第二位置差以及所述上侧嘴唇关键点与所述下侧嘴唇关键点在y轴方向上的第三位置差；
其中，计算所述第一位置差需要从所述左侧嘴唇关键点中提取任意两个关键点，提取出的这两个关键点在x轴方向上的位置差即为所述第一位置差。例如，在左侧嘴唇关键点中提取嘴唇关键点49和50，则所述第一位置差为嘴唇关键点49和50在x轴方向上的位置差；
计算所述第二位置差需要从所述右侧嘴唇关键点中提取与已提取的两个左侧关键点相对应的两 个关键点，提取出的这两个关键点在x轴方向上的位置差即为所述第二位置差。例如，若左侧嘴唇关键点提取了49和50，则在右侧嘴唇关键点中提取对应的嘴唇关键点55和54，则所述第二位置差为嘴唇关键点55和54在x轴方向上的位置差；
计算所述第三位置差需要提取一个上侧嘴唇关键点以及与其对应的一个下侧嘴唇关键点，提取出的这两个关键点在y轴方向上的位置差即为所述第三位置差。例如在上侧嘴唇关键点中提取63，则在下侧嘴唇关键点中提取67，则所述第三位置差为嘴唇关键点63和67在y轴方向上的位置差。
根据人像面部图像中人脸区域的宽度、高度、图像分辨率、所述第一位置差以及所述第二位置差，确定人像面部朝向；其中，所述人像面部朝向包括：正脸朝向、左偏朝向以及右偏朝向；
根据人像面部图像中人脸区域的宽度、高度、图像分辨率以及所述第三位置差，确定嘴唇形态。
在一些实施方式中，判断面部朝向的方法可以为：
设d 1＝P51. x-P50. x，d 2＝P54. x-P53. x，，
Figure PCTCN2022114755-appb-000001
其中，d 1为关键点51和50在x轴方向上的位置差，即所述第一位置差，d 2为关键点54和53在x轴方向上的位置差，即所述第二位置差，R w为人脸区域的宽，R h为人脸区域的高，WH为图像分辨率，P52. x-P51. x为关键点52和51在x轴方向上的位置差。
若d 1-d 2＞δ x，则判定面部朝向为右偏朝向，若d 2-d 1＞δ x，则判定面部朝向为左偏朝向；否则，判断面部朝向为正脸朝向。
判断嘴唇形态的方法可以为：
设open y＝P67. y-P63. y，
Figure PCTCN2022114755-appb-000002
其中，open y为关键点67和63在y轴方向上的位置差，即所述第三位置差；P58. y-P67. y为关键点67和58在y轴方向上的位置差；此外，设第一预设程度阈值Y 1，第二预设程度阈值Y 2，且Y 2＞Y 1。
若open y＞δ y，则判定嘴唇形态为张嘴；若open y＜δ y，则判定嘴唇形态为未张嘴；
当嘴唇形态为张嘴时，若open y＜Y 1，则判定嘴唇形态为张嘴程度较小，即张嘴程度小于第一预设程度阈值；若open y＞Y 2，则判定嘴唇形态为张嘴程度较大，即张嘴程度大于第二预设程度阈值。
步骤S102，根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置。
在一些实施方式中，在所述根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置之前，还包括：
根据嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整。
当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，若所述嘴角关键点不符合第一预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设条件为，在灰度色彩空间中，所述嘴角关键点中间的像素点的亮度低于第一预设阈值；
当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点不符合第二预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值；所述第二预设条件为，所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。
在一些实施方式中，当人像没有张嘴或张嘴程度较小时，若人像未改变朝向或嘴唇没有改变形状，则两嘴角关键点中间点应处于上下嘴唇交界处，相较于四周，所述中间点处于凹陷处的阴影区域。因此在灰度色彩空间中，若该点的亮度比周围宏块的亮度更低则说明它处于所述阴影区域，即可以判断人像未改变朝向或嘴唇没有改变形状。反之，若人像改变了朝向或嘴唇形状发生改变，则该中心点不处于该所述阴影区域，因此在灰度色彩空间中，该中心点的亮度不一定比周围宏块的亮度更低。
因此，第一预设条件可为：在灰度色彩空间中，所述嘴角关键点中间的像素点的亮度低于第一预设阈值。那么当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，若在灰度色彩 空间中，所述嘴角关键点中间的像素点的亮度不低于第一预设阈值，则对所述嘴唇关键点的位置进行调整；
在一些实施方式中，所述第一预设阈值可以为，嘴唇区域中亮度最暗的10％。
在一些实施方式中，当人像张嘴正常或张嘴程度较大时，若人像未改变朝向或嘴唇没有改变形状，则两嘴角关键点处于周围宏块的边缘附近，同时，嘴角关键点只位于肤色区域或是唇色区域；反之，若人像改变了朝向或嘴唇形状发生了改变，则嘴角关键点所处位置也发生了变化，即将不再处于周围宏块的边缘，且其所在的肤色区域和唇色区域可能发生重合，则需要对嘴角关键点的位置进行调整。
因此，第二预设条件可为：所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。那么当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中，则不对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值。
步骤S103，检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号。
在一些实施方式中，根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数，根据所述皮肤色调判断系数判断所述人像面部图像中的人脸区域的皮肤色调；根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
步骤S104，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色。
在一些实施方式中，根据预设的配色系数对口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系；
依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找所述人脸区域的皮肤色调对应的目标口红颜色区间；
根据所述人脸区域的皮肤色号，在目标口红颜色区间中选择对应的口红色号。
在一些实施方式中，所述预设的配色系数可以为：
Figure PCTCN2022114755-appb-000003
其中，μ L为人脸区域在Lab色彩空间中亮度L的均值；ξ stone为人脸区域的皮肤色调判断系数；η level为人脸区域的皮肤色号判断系数；L max为人脸区域在Lab色彩空间中亮度L的最大值；L min为人脸区域在Lab色彩空间中亮度L的最小值。
将口红颜色样本库中的口红颜色依据配色系数Φ进行区间划分，划分出从蓝调调色到黄调调色的口红颜色区间。
步骤S105，依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
在一些实施方式中，依据经过确定的所述嘴唇关键点的位置，准确描述出人像面部图像中的嘴唇图像信息，并基于此对人像嘴唇进行融合上色；其中，融合上色所用颜色为步骤S104确定出的口红色号对应的口红颜色。
根据本实施例提供的人像唇部的自动上妆方法，从人像面部图像中提取嘴唇关键点，并检测人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置。同时，检测人像面部图像中的人脸区域的皮肤色调和皮肤色号，从口红颜色样本库中选择目标口红颜色。最终，依据嘴唇关键点的位置和选择的口红颜色对人像的嘴唇进行融合上色。利用本发发明提供的技术方案，能够基于从人像面部图像中提取的嘴唇关键点，能够检测人像的面部朝向和嘴唇形态，并调整人像中嘴唇关键点的位置，修正由不同的面部朝向和嘴唇形态带来的关键点位置变化，使调整后的关键点位置符合当前人像的面部朝向和嘴唇形态；并基于检测到的皮肤色调和皮肤色号，自动选择出适合的口红色号对人像嘴唇进行融合上色。有效解决了当人像面部朝向和嘴唇形态发生变化时，关键点无法调整导致的上色位置不准确的问题，并通过自动选择口红颜色简化了用户的操作步骤，即优化了上妆效果，同时降低了操作难度，有效提高了用户体验。
根据上述方法，图2为本公开实施例提供的人像唇部的自动上妆方法中调整嘴唇关键点位置的流程示意图，如图2所示，该方法包括如下步骤：
步骤S201，分别确定唇色区域的唇色判断准则和肤色区域的肤色判断准则。
在一些实施方式中，在唇色区域计算Lab色彩空间的a-b的均值方差比，作为唇色判断准则；在脸颊区域找任意区域，计算Lab色彩空间的a-b的均值方差比，作为肤色判断准则。
在一些实施方式中，依据关键点57，58，59，66，67，68确定唇色区域。
步骤S202，检测人像面部图像中的人像面部朝向以及嘴唇形态。
检测人像面部图像中的人像面部朝向以及嘴唇形态的方式已在前述实施例的步骤S101中进行了详细阐述。
其中，面部朝向分为正脸朝向、左偏朝向、右偏朝向；嘴唇形态分为：张嘴、未张嘴。
步骤S203，根据人像面部图像中的人像面部朝向以及嘴唇形态选择相应的调整方式调整嘴唇关键点的位置。
在一些实施方式中，针对所述嘴唇关键点中的侧边嘴角关键点：
当所述人像面部朝向为正脸朝向且所述嘴唇形态为未张嘴形态时，在一侧正脸调整区域中查找最暗的两个像素点，将所述两个像素点的中间位置作为该侧侧边嘴角关键点调整后的位置；其中，所述正脸调整区域是根据所述侧边嘴角关键点的位置向四周扩展第一预设数量的像素点所形成的区域。由于人像处于正脸朝向且未张嘴形态，则一侧正脸调整区域中最暗处即为因凹陷而处于阴影的嘴角处，则最暗的两个像素点的中间位置可以较为准确的确定该侧的嘴角位置；同时，根据侧边嘴角关键点位置向四周扩展第一预设数量的像素点确定正脸调整区域可以基于嘴角关键点位置确定区域的大致位置，且通过预设扩展的范围可在保证精确性的情况下保证足够的调整余地。
在一些实施方式中，所述第一预设数量可以为10，即根据所述侧边嘴角关键点的位置向四周扩展10个像素点所形成的区域为正脸调整区域。
当所述人像面部朝向为正脸朝向且所述嘴唇形态为张嘴形态时，对正脸调整区域进行边缘检测，依据第一边缘检测结果确定第一参照点，以所述第一参照点为中心，查找肤色区域和唇色区域的第一肤色分界点和第一唇色分界点，将所述第一肤色分界点和所述第一唇色分界点的中间位置作为侧边嘴角关键点调整后的位置。通过边缘检测确定出原嘴角区域的第一参照点，又由于嘴角位置即为肤色和唇色的分界处，则基于第一参照点确定出第一肤色分界点和第一唇色分界点，二者的中间位置可以准确地表示侧边嘴角关键点调整后的位置。
其中，所述第一肤色分界点是根据肤色区域的肤色判断准则确定，所述第一唇色分界点根据唇色判断准则确定；
侧边嘴角关键点包括：左侧嘴角关键点和右侧嘴角关键点，其中，左侧嘴角关键点可包括49和61号关键点，右侧嘴角关键点可包括55和65号关键点。对侧边嘴角关键点的调整包括对左侧嘴角关键点的调整以及对右侧嘴角关键点的调整。
在一些实施方式中，所述对正脸调整区域进行边缘检测可以为对正脸调整区域进行灰度图像的拉普拉斯边缘检测；对于左侧嘴角关键点的调整，可从第一边缘检测结果中的边缘点中选择距离49和61号关键点最近的点作为所述第一参照点；同理，对于右侧嘴角关键点的调整，可从第一边缘检测结果中的边缘点中选择距离55和65号关键点最近的点作为所述第一参照点。
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为未张嘴形态时，在侧偏调整区域中查找最暗的像素点，依据所述最暗的像素点在侧偏调整区域中的相对位置，调整侧边嘴角关键点的位置；其中，所述侧偏调整区域是根据所述侧边嘴角关键点的位置向左右方向扩展第二预设数量的像素点且向上下方向扩展第三预设数量的像素点所形成的区域。由于人像面部朝向发生改变，左偏朝向时，各个关键点也发生了移动，通过查找最暗点，并确定最暗点同原嘴角关键点的位置关系，可以更为精确地确定关键点位置。
在一些实施方式中，所述侧偏调整区域可以分为左侧嘴角调整区域和右侧嘴角调整区域，依据左侧嘴角关键点确定出左侧嘴角调整区域，依据右侧嘴角关键点确定出右侧嘴角调整区域；在确定左侧嘴角调整区域时，第二预设数量可以为P61. x-P49. x，即关键点61与关键点49之间的像素个数；所述第三预设数量可以为10；在确定右侧嘴角调整区域时，第二预设数量可以为P65. x-P55. x，即关键点65与关键点55之间的像素个数；所述第三预设数量可以为10。
即左侧嘴角调整区域为以49号关键点向左扩展P61. x-P49. x个像素，以61号关键点向右扩展P61. x-P49. x个像素，并且以49和61号关键点上下扩展10个像素所形成的区域；右侧嘴角调整区域为以55号关键点向右扩展P65. x-P55. x个像素，以65号关键点向左扩展P65. x-P55. x个像素，上下扩展10个像素所形成的区域。
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为张嘴形态时，在侧偏调整区域中查找最暗的像素点，并对侧偏调整区域进行边缘检测，依据第二边缘检测结果确定第二参照点，以所述第二参照点为中心，查找肤色区域和唇色区域的第二肤色分界点和第二唇色分界点，依据最 暗的像素点在侧偏调整区域中的相对位置、所述第二肤色分界点以及所述第二唇色分界点，调整侧边嘴角关键点的位置。由于人像面部朝向发生改变且为张嘴形态，因此，基于嘴角较暗，通过边缘检测确定出边缘上的最暗点，再基于该最暗点同原嘴角关键点的位置关系，利用最暗点和肤色分界点或唇色分界点，精确地确定出调整后的嘴角关键点。
其中，所述第二肤色分界点根据肤色区域的肤色判断准则确定，所述第二唇色分界点根据唇色判断准则确定；
在一些实施方式中，所述对侧偏调整区域进行边缘检测可以对侧偏调整区域进行灰度图像的拉普拉斯边缘检测；对于左侧嘴角关键点的调整，可从第二边缘检测结果中的边缘点中距离49和61号关键点最近的点作为所述第二参照点；同理，对于右侧嘴角关键点的调整，可从第二边缘检测结果中的边缘点中选择距离55和65号关键点最近的点作为所述第二参照点。
在一些实施方式中，当人像面部朝向为左偏朝向或右偏朝向时，所述侧偏调整区域可以分为左侧嘴角调整区域和右侧嘴角调整区域，依据左侧嘴角关键点确定出左侧嘴角调整区域，依据右侧嘴角关键点确定出右侧嘴角调整区域；所述左侧嘴角调整区域为以49号关键点向左扩展P61. x-P49. x个像素，以61号关键点向右扩展P61. x-P49. x个像素，并且以49和61号关键点上下扩展10个像素所形成的区域；所述右侧嘴角调整区域为以55号关键点向右扩展P65. x-P55. x个像素，以65号关键点向左扩展P65. x-P55. x个像素，上下扩展10个像素所形成的区域。
当人像面部朝向为左偏朝向时，若左侧嘴角调整区域中最暗的像素点在49和61号关键点之间，则以49和61号关键点的中间位置作为左侧嘴角关键点调整后的位置；若左侧嘴角调整区域中最暗的像素点在49号关键点的左侧，则将所述最暗点向左扩展到第二肤色分界点，以所述最暗点和第二肤色临界点的中间位置作为左侧嘴角关键点调整后的位置；若左侧嘴角调整区域中最暗的像素点在61号关键点右侧，则将所述最暗点向右扩展到第二唇色分界点，以所述最暗点和第二唇色临界点的中间位置作为左侧嘴角关键点调整后的位置。
若右侧嘴角调整区域中最暗的像素点在55和65号关键点之间，则以55和65号关键点的中间位置作为右侧嘴角关键点调整后的位置；若右侧嘴角调整区域中最暗的像素点在55号关键点右侧，则将所述最暗点向右扩展到第二肤色分界点，以所述最暗点和第二肤色临界点的中间位置作为右侧嘴角关键点调整后的位置；若右侧嘴角调整区域中最暗的像素点在65号关键点左侧，则将所述最暗点向左扩展到第二唇色分界点，以所述最暗点和第二唇色临界点的中间位置作为右侧嘴角关键点调整后的位置。
同理，当人像面部朝向为右偏朝向，分别确定左侧嘴角关键点调整后的位置以及右侧嘴角关键点调整后的位置。
根据上述实施方式，图3为本公开实施例提供的人像唇部的自动上妆方法中自动选择口红颜色的流程示意图，如图3所示，该方法包括如下步骤：
步骤S301，确定人像面部图像中的人脸区域的皮肤色调。
在一些实施方式中，所述人像面部图像中的人脸区域的皮肤色调包括：冷色调、暖色调、中性色调；根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数；当所述皮肤色调判断系数处于第一预设区间之内时，则判定所述皮肤色调为冷色调；当所述皮肤色调判断系数处于第二预设区间之内时，则判定所述皮肤色调为暖色调；当所述皮肤色调判断系数不处于第一预设区间且不处于第二预设区间之内时，则判定所述皮肤色调为中性色调。
在一些实施方式中，将人像面部图像从RGB色彩空间变换到HSI色彩空间；其中，H为色调(Hue)，S为饱和度(Saturation或Chroma)，I为亮度(Intensity或Brightness)。
根据人像面部图像中的人脸区域的色调H、饱和度S和亮度I，计算色调H的均值、方差、最大值、最小值；计算饱和度S的均值、方差。
在人像面部图像中的眼睛区域，通过将眼球区域的颜色，变换到Lab色彩空间，计算眼球区域的b的均值；其中，b为Lab色彩空间内黄色到蓝色之间的位置坐标；若b>0时，说明该区域颜色接近黄色，皮肤为暖色。
在一些实施方式中，所述皮肤色调判断系数的计算方式为：
Figure PCTCN2022114755-appb-000004
其中，H为人脸区域在HSI色彩区域中的色调，当H大于1.5π时，为H重新赋值为H-2π；其他情况，H取值即为H原值。
μ H为人脸区域的色调H的均值；μ S为人脸区域的饱和度S的均值；μ b为眼球区域的b坐标的均值；σ H为人脸区域的色调H的方差；σ S为人脸区域的饱和度S的方差；H max为人脸区域的色调H的最大值；H min为人脸区域的色调H的最小值。
当ξ stone在
Figure PCTCN2022114755-appb-000005
之间时，则判定所述皮肤色调为冷色调；当ξ stone在
Figure PCTCN2022114755-appb-000006
之间时，则判定所述皮肤色调为暖色调；其他情况，则判定所述皮肤色调为中性色调。
步骤S302，确定人像面部图像中的人脸区域的皮肤色号。
在一些实施方式中，根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
在一些实施方式中，计算HSI色彩空间中，计算色调H的均值、方差；计算亮度I的均值、方差、最大值、最小值；
所述皮肤色号判断系数的计算方式为：
Figure PCTCN2022114755-appb-000007
其中，H为人脸区域在HSI色彩区域中的色调，当H大于1.55π时，为H重新赋值为H-2π；其他情况，H取值即为H原值。
μ I为人脸区域的亮度I的均值；μ H为人脸区域的色调H的均值；σ H为人脸区域的色调H的方差；σ I为人脸区域的亮度I的方差；I max为人脸区域的亮度I的最大值；I min为人脸区域的亮度I的最小值。
在一些实施方式中，预设所述皮肤色号可包括：第一色号、第二色号和第三色号；
当η level＜TH 0时，则判定所述人脸区域的皮肤色号为第一色号；当η level＞TH 1时，则判定所述人脸区域的皮肤色号为第三色号；其他情况，则判定所述人脸区域的皮肤色号为第二色号；其中，TH 0、TH 1为预设的用于区分色号的色号阈值。
步骤S303，对口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系。
在一些实施方式中，根据预设的配色系数对口红颜色样本库中所有口红颜色进行区间划分，再配置各个口红颜色区间与预设皮肤色调之间的对应关系；其中，所述预设皮肤色调包括：冷色调、暖色调、中性色调。
在一些实施方式中，所述配色系数预设的配色系数可以为
Figure PCTCN2022114755-appb-000008
其中，ξ stone为人脸区域的皮肤色调判断系数；η level为人脸区域的皮肤色号判断系数；μ L为人脸区域在Lab色彩空间中亮度L的均值；L max为人脸区域在Lab色彩空间中亮度L的最大值；L min为人脸区域在Lab色彩空间中亮度L的最小值。
将口红颜色样本库中的口红颜色依据配色系数Φ进行区间划分，划分出的从蓝调调色到黄调调色的口红颜色区间；
在一些实施方式中，各个口红颜色区间与预设皮肤色调之间的对应关系可以是：
冷色调对应蓝绿色系的口红颜色区间，暖色调对应红黄色系的口红颜色区间，中性色调则可以对应全部色系的口红颜色区间。
步骤S304，依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找人脸区域的皮肤色调对应的目标口红颜色区间。
在一些实施方式中，依据步骤S301中确定的人像面部图像中的人脸区域的皮肤色调，以及步骤S304中确定的各个口红颜色区间与预设皮肤色调之间的对应关系，查找出所述人像面部图像对应的口红颜色区间。
步骤S305，根据人脸区域的皮肤色号，在目标口红颜色区间中选择对应的口红色号。
在一些实施方式中，根据步骤S302中确定的人像面部图像中的人脸区域的皮肤色号从步骤S304查找出的口红颜色区间中选择出在一些实施方式中适合所述人像面部图像肤色黄白程度的口 红色号。
图4为本公开实施例提供的人像唇部的自动上妆装置的结构框图，如图4所示，该装置包括：关键点确定模块401、颜色选择模块402和融合上色模块403；其中，
所述关键点确定模块401，配置为从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置。
在一些实施方式中，所述关键点确定模块401，还配置为：识别人像面部图像中的面部轮廓及五官，为人像面部图像设置关键点，从全部关键点中提取嘴唇关键点；其中，所述嘴唇关键点用于描述嘴唇图像信息。
在一些实施方式中，所述关键点确定模块401，还配置为：根据嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整。
在一些实施方式中，所述关键点确定模块401，还配置为：当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，若所述嘴角关键点不符合第一预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设条件为在灰度色彩空间中所述嘴角关键点中间的像素点的亮度低于第一预设阈值；
当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点不符合第二预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值；所述第二预设条件为所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。
在一些实施方式中，所述关键点确定模块401，还配置为：针对所述嘴唇关键点中的侧边嘴角关键点，当所述人像面部朝向为正脸朝向且所述嘴唇形态为未张嘴形态时，在正脸调整区域中查找最暗的两个像素点，将所述两个像素点的中间位置作为侧边嘴角关键点调整后的位置；其中，所述正脸调整区域是根据所述侧边嘴角关键点的位置向四周扩展第一预设数量的像素点所形成的区域；
当所述人像面部朝向为正脸朝向且所述嘴唇形态为张嘴形态时，对正脸调整区域进行边缘检测，依据第一边缘检测结果确定第一参照点，以所述第一参照点为中心，查找肤色区域和唇色区域的第一肤色分界点和第一唇色分界点，将所述第一肤色分界点和所述第一唇色分界点的中间位置作为侧边嘴角关键点调整后的位置；
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为未张嘴形态时，在侧偏调整区域中查找最暗的像素点，依据所述最暗的像素点在侧偏调整区域中的相对位置，调整侧边嘴角关键点的位置；其中，所述侧偏调整区域是根据所述侧边嘴角关键点的位置向左右方向扩展第二预设数量的像素点且向上下方向扩展第三预设数量的像素点所形成的区域；
当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为张嘴形态时，在侧偏调整区域中查找最暗的像素点，并对侧偏调整区域进行边缘检测，依据第二边缘检测结果确定第二参照点，以所述第二参照点为中心，查找肤色区域和唇色区域的第二肤色分界点和第二唇色分界点，依据最暗的像素点在侧偏调整区域中的相对位置、所述第二肤色分界点以及所述第二唇色分界点，调整侧边嘴角关键点的位置。
所述颜色选择模块402，配置为检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色。
在一些实施方式中，所述颜色选择模块402，还配置为：根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数；当所述皮肤色调判断系数处于第一预设区间之内时，则判定所述皮肤色调为冷色调；当所述皮肤色调判断系数处于第二预设区间之内时，则判定所述皮肤色调为暖色调；当所述皮肤色调判断系数不处于第一预设区间且不处于第二预设区间之内时，则判定所述皮肤色调为中性色调；根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
在一些实施方式中，所述颜色选择模块402，还配置为：根据预设的配色系数对口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系；依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找所述人脸区域的皮肤色调对应的目标口红颜色区间；根据所述人脸区域的皮肤色号，在目标口红颜色区间中选择对应的口红色号。
所述融合上色模块403，配置为依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
在一些实施方式中，所述融合上色模块403，还配置为：根据关键点确定模块401确定出的嘴唇关键点的位置以及颜色选择模块402选择出的对应的口红色号，对人像面部图像的嘴唇区域进行 上色。
根据本实施例提供的人像唇部的自动上妆装置，从人像面部图像中提取嘴唇关键点，并检测人像面部朝向以及嘴唇形态，调整所述嘴唇关键点的位置。同时，检测人像面部图像中的人脸区域的皮肤色调和皮肤色号，从口红颜色样本库中选择目标口红颜色。最终，依据嘴唇关键点的位置和选择的口红颜色对人像的嘴唇进行融合上色。利用本公开实施例提供的技术方案，能够基于从人像面部图像中提取的嘴唇关键点，检测人像的面部朝向和嘴唇形态，并调整人像中嘴唇关键点的位置，修正由不同的面部朝向和嘴唇形态带来的关键点位置变化，使调整后的关键点位置符合当前人像的面部朝向和嘴唇形态；并基于检测到的皮肤色调和皮肤色号，自动选择出适合的口红色号对人像嘴唇进行融合上色。有效解决了当人像面部朝向和嘴唇形态发生变化时，关键点无法调整导致的上色位置不准确的问题，并通过自动选择口红颜色简化了用户的操作步骤，即优化了上妆效果，同时降低了操作难度，有效提高了用户体验。
本公开实施例还提供了一种非易失性计算机存储介质，计算机存储介质存储有一可执行指令，可执行指令可执行上述任意方法实施例中的人像唇部的自动上妆方法。
图5为本公开实施例提供的一种计算设备的结构示意图，本公开实施例不对计算设备的实现做限定。
如图5所示，该计算设备可以包括：处理器(processor)502、通信接口(Communications Interface)504、存储器(memory)506、以及通信总线508。
其中：
处理器502、通信接口504、以及存储器506通过通信总线508完成相互间的通信。
通信接口504，配置为与其它设备比如客户端或其它服务器等的网元通信。
处理器502，配置为执行程序510，可以执行上述人像唇部的自动上妆方法实施例中的相关步骤。
在一些实施方式中，程序510可以包括程序代码，该程序代码包括计算机操作指令。
处理器502可能是中央处理器CPU，或者是特定集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本公开实施例的一个或多个集成电路。计算设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。
存储器506，配置为存放程序510。存储器506可能包含高速RAM存储器，也可能还包括非易失性存储器(non-volatile memory)，例如一个磁盘存储器。
程序510配置为使处理器502执行上述任意方法实施例中的人像唇部的自动上妆方法。程序510中各步骤的实现可以参见上述人像唇部的自动上妆方法实施例中的相应步骤和单元中对应的描述。所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的设备和模块的工作过程，可以参考前述方法实施例中的对应过程描述。
本公开实施例提供了一种计算机程序产品，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行上述任意方法实施例中的人像唇部的自动上妆方法。
在此提供的算法和显示不与任何特定计算机、虚拟系统或者其它设备固有相关。各种通用系统也可以与基于在此的示教一起使用。根据上面的描述，构造这类系统所要求的结构是显而易见的。此外，本公开也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本公开的内容，并且上面对特定语言所做的描述是为了披露本公开的最佳实施方式。
在此处所提供的说明书中，说明了大量细节。然而，能够理解，本公开可以在没有这些细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本公开的示例性实施例的描述中，本公开的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本公开要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本公开的单独实施例。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过 程或者单元中的一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本公开的范围之内并且形成不同的实施例。例如，在权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
本公开的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本公开的一些或者全部部件的一些或者全部功能。本公开还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本公开的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
应该注意的是上述实施例对本公开进行说明而不是对本公开进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本公开可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来体现。单词第一、第二、以及第三等的使用不表示任何顺序，可将这些单词解释为名称。
工业实用性
通过从人像面部图像提取嘴唇关键点，并根据检测到的人像面部朝向及嘴唇形态，调整所述嘴唇关键点的位置，确保了调整后嘴唇关键点位置的准确性；然后再根据检测到的人像面部图像的皮肤色调和皮肤色号自动选择合适的口红颜色并上色。由此解决了相关技术中按照人脸检测关键点标注的基础上进行曲线拟合并上色，在人像光线、角度等发生变化时关键点位置不准确、上妆有瑕疵的问题，同时，省去了用户自己选择口红颜色的步骤，根据相应条件自动选择口红颜色并上色，更符合人类审美，提高了上妆的效率以及用户体验。

Claims (17)

  1. 一种人像唇部的自动上妆方法，包括：
    从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据所述人像面部朝向以及所述嘴唇形态，调整所述嘴唇关键点的位置；
    检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色；
    依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
  2. 根据权利要求1所述的方法，其中，所述从人像面部图像中提取嘴唇关键点，包括：
    识别人像面部图像中的面部轮廓及五官，为人像面部图像设置关键点，从全部关键点中提取嘴唇关键点；其中，所述嘴唇关键点用于描述嘴唇图像信息。
  3. 根据权利要求1所述的方法，其中，在所述根据所述人像面部朝向以及所述嘴唇形态，调整所述嘴唇关键点的位置之前，所述方法还包括：
    根据所述嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整。
  4. 根据权利要求3所述的方法，其中，所述根据所述嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整，包括：
    当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，若所述嘴角关键点不符合第一预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设条件为在灰度色彩空间中所述嘴角关键点中间的像素点的亮度低于第一预设阈值；
    当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点不符合第二预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值；所述第二预设条件为所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。
  5. 根据权利要求1所述的方法，其中，所述根据所述人像面部朝向以及所述嘴唇形态，调整所述嘴唇关键点的位置，包括：
    针对所述嘴唇关键点中的侧边嘴角关键点，当所述人像面部朝向为正脸朝向且所述嘴唇形态为未张嘴形态时，在正脸调整区域中查找最暗的两个像素点，将所述两个像素点的中间位置作为侧边嘴角关键点调整后的位置；其中，所述正脸调整区域是根据所述侧边嘴角关键点的位置向四周扩展第一预设数量的像素点所形成的区域；
    当所述人像面部朝向为正脸朝向且所述嘴唇形态为张嘴形态时，对正脸调整区域进行边缘检测，依据第一边缘检测结果确定第一参照点，以所述第一参照点为中心，查找肤色区域和唇色区域的第一肤色分界点和第一唇色分界点，将所述第一肤色分界点和所述第一唇色分界点的中间位置作为侧边嘴角关键点调整后的位置；
    当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为未张嘴形态时，在侧偏调整区域中查找最暗的像素点，依据所述最暗的像素点在侧偏调整区域中的相对位置，调整侧边嘴角关键点的位置；其中，所述侧偏调整区域是根据所述侧边嘴角关键点的位置向左右方向扩展第二预设数量的像素点且向上下方向扩展第三预设数量的像素点所形成的区域；
    当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为张嘴形态时，在侧偏调整区域中查找最暗的像素点，并对侧偏调整区域进行边缘检测，依据第二边缘检测结果确定第二参照点，以所述第二参照点为中心，查找肤色区域和唇色区域的第二肤色分界点和第二唇色分界点，依据最暗的像素点在侧偏调整区域中的相对位置、所述第二肤色分界点以及所述第二唇色分界点，调整侧边嘴角关键点的位置。
  6. 根据权利要求1-5任一项所述的方法，其中，所述检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，包括：
    根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数；当所述皮肤色调判断系数处于第一预设区间之内时，则判定所述皮肤色调为冷色调；当所述皮肤色调判断系数处于第二预设区间之内时，则判定所述皮肤色调为暖色调；当所述皮肤色调判断系数不处于第一预设区间且不处于第二预设区间之内时，则判定所述皮肤色调为中性色调；
    根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
  7. 权利要求1-5任一项所述的方法，其中，所述根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色，包括：
    根据预设的配色系数对所述口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系；
    依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找所述人脸区域的皮肤色调对应的目标口红颜色区间；
    根据所述人脸区域的皮肤色号，在所述目标口红颜色区间中选择对应的口红色号。
  8. 一种人像唇部的自动上妆装置，包括：关键点确定模块、颜色选择模块以及融合上色模块；其中，
    所述关键点确定模块，配置为从人像面部图像中提取嘴唇关键点，并检测人像面部图像中的人像面部朝向以及嘴唇形态，根据所述人像面部朝向以及所述嘴唇形态，调整所述嘴唇关键点的位置；
    所述颜色选择模块，配置为检测所述人像面部图像中的人脸区域的皮肤色调和皮肤色号，根据所述皮肤色调和所述皮肤色号，从口红颜色样本库中选择目标口红颜色；
    所述融合上色模块，配置为依据所述嘴唇关键点的位置使用所述目标口红颜色进行融合上色。
  9. 根据权利要求8所述的装置，其中，所述关键点确定模块，还配置为：
    识别人像面部图像中的面部轮廓及五官，为人像面部图像设置关键点，从全部关键点中提取嘴唇关键点；其中，所述嘴唇关键点用于描述嘴唇图像信息。
  10. 根据权利要求8所述的装置，其中，所述关键点确定模块在根据所述人像面部朝向以及所述嘴唇形态，调整所述嘴唇关键点的位置之前，所述关键点确定模块，还配置为：
    根据所述嘴唇形态以及所述嘴唇关键点中的嘴角关键点，判断是否对所述嘴唇关键点的位置进行调整。
  11. 根据权利要求10所述的装置，其中，所述关键点确定模块，还配置为：
    当所述嘴唇形态为未张嘴形态或张嘴程度小于第一预设程度阈值时，若所述嘴角关键点不符合第一预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设条件为在灰度色彩空间中所述嘴角关键点中间的像素点的亮度低于第一预设阈值；
    当所述嘴唇形态为张嘴形态且张嘴程度大于第二预设程度阈值时，若所述嘴角关键点不符合第二预设条件，则对所述嘴唇关键点的位置进行调整；其中，所述第一预设程度阈值小于或等于所述第二预设程度阈值；所述第二预设条件为所述嘴角关键点位于嘴唇区域的边缘附近且未同时位于肤色区域或唇色区域中。
  12. 根据权利要求8所述的装置，其中，所述关键点确定模块，还配置为：
    针对所述嘴唇关键点中的侧边嘴角关键点，当所述人像面部朝向为正脸朝向且所述嘴唇形态为未张嘴形态时，在正脸调整区域中查找最暗的两个像素点，将所述两个像素点的中间位置作为侧边嘴角关键点调整后的位置；其中，所述正脸调整区域是根据所述侧边嘴角关键点的位置向四周扩展第一预设数量的像素点所形成的区域；
    当所述人像面部朝向为正脸朝向且所述嘴唇形态为张嘴形态时，对正脸调整区域进行边缘检测，依据第一边缘检测结果确定第一参照点，以所述第一参照点为中心，查找肤色区域和唇色区域的第一肤色分界点和第一唇色分界点，将所述第一肤色分界点和所述第一唇色分界点的中间位置作为侧边嘴角关键点调整后的位置；
    当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为未张嘴形态时，在侧偏调整区域中查找最暗的像素点，依据所述最暗的像素点在侧偏调整区域中的相对位置，调整侧边嘴角关键点的位置；其中，所述侧偏调整区域是根据所述侧边嘴角关键点的位置向左右方向扩展第二预设数量的像素点且向上下方向扩展第三预设数量的像素点所形成的区域；
    当所述人像面部朝向为左偏朝向或者右偏朝向且所述嘴唇形态为张嘴形态时，在侧偏调整区域中查找最暗的像素点，并对侧偏调整区域进行边缘检测，依据第二边缘检测结果确定第二参照点，以所述第二参照点为中心，查找肤色区域和唇色区域的第二肤色分界点和第二唇色分界点，依据最暗的像素点在侧偏调整区域中的相对位置、所述第二肤色分界点以及所述第二唇色分界点，调整侧边嘴角关键点的位置。
  13. 根据权利要求8-12任一项所述的方法，其中，所述颜色选择模块，还配置为：
    根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色调判断系数；当所述皮 肤色调判断系数处于第一预设区间之内时，则判定所述皮肤色调为冷色调；当所述皮肤色调判断系数处于第二预设区间之内时，则判定所述皮肤色调为暖色调；当所述皮肤色调判断系数不处于第一预设区间且不处于第二预设区间之内时，则判定所述皮肤色调为中性色调；
    根据所述人像面部图像中的人脸区域的颜色参数，计算出对应的皮肤色号判断系数；根据所述皮肤色号判断系数判断所述人脸区域的皮肤色号。
  14. 根据权利要求8-12任一项所述的装置，其中，所述颜色选择模块，还配置为：
    根据预设的配色系数对所述口红颜色样本库中所有口红颜色进行区间划分，并配置各个口红颜色区间与预设皮肤色调之间的对应关系；
    依据各个口红颜色区间与预设皮肤色调之间的对应关系，查找所述人脸区域的皮肤色调对应的目标口红颜色区间；
    根据所述人脸区域的皮肤色号，在所述目标口红颜色区间中选择对应的口红色号。
  15. 一种计算设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
    所述存储器配置为存放一可执行指令，所述可执行指令使所述处理器执行如权利要求1-7中任一项所述的人像唇部的自动上妆方法对应的操作。
  16. 一种计算机存储介质，所述存储介质中存储有一可执行指令，所述可执行指令使处理器执行如权利要求1-7中任一项所述的人像唇部的自动上妆方法对应的操作。
  17. 一种计算机程序产品，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行如权利要求1-7中任一项所述的人像唇部的自动上妆方法对应的操作。
PCT/CN2022/114755 2021-08-25 2022-08-25 人像唇部的自动上妆方法、装置、设备、存储介质及程序产品 WO2023025239A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110982907.2A CN113674177B (zh) 2021-08-25 2021-08-25 一种人像唇部的自动上妆方法、装置、设备和存储介质
CN202110982907.2 2021-08-25

Publications (1)

Publication Number Publication Date
WO2023025239A1 true WO2023025239A1 (zh) 2023-03-02

Family

ID=78546249

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/114755 WO2023025239A1 (zh) 2021-08-25 2022-08-25 人像唇部的自动上妆方法、装置、设备、存储介质及程序产品

Country Status (2)

Country Link
CN (1) CN113674177B (zh)
WO (1) WO2023025239A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113674177B (zh) * 2021-08-25 2024-03-26 咪咕视讯科技有限公司 一种人像唇部的自动上妆方法、装置、设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08202754A (ja) * 1995-01-27 1996-08-09 Pola Chem Ind Inc リップカラーアドバイスシステム及び方法
JP2007097950A (ja) * 2005-10-06 2007-04-19 Shiseido Co Ltd 唇の化粧方法
JP2009163465A (ja) * 2007-12-29 2009-07-23 Yurakusha:Kk 似顔絵イラストデータ提供システム
CN108804975A (zh) * 2017-04-27 2018-11-13 丽宝大数据股份有限公司 唇彩指引装置及方法
CN113674177A (zh) * 2021-08-25 2021-11-19 咪咕视讯科技有限公司 一种人像唇部的自动上妆方法、装置、设备和存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006120005A (ja) * 2004-10-22 2006-05-11 Shiseido Co Ltd 唇の形状分類方法及び分類マップ
JP4831259B1 (ja) * 2011-03-10 2011-12-07 オムロン株式会社 画像処理装置、画像処理方法、および制御プログラム
CN106909870A (zh) * 2015-12-22 2017-06-30 中兴通讯股份有限公司 人脸图像的检索方法及装置
CN107800966B (zh) * 2017-10-31 2019-10-18 Oppo广东移动通信有限公司 图像处理的方法、装置、计算机可读存储介质和电子设备
CN107862657A (zh) * 2017-10-31 2018-03-30 广东欧珀移动通信有限公司 图像处理方法、装置、计算机设备及计算机可读存储介质
CN109685713B (zh) * 2018-11-13 2024-05-10 平安科技（深圳）有限公司 化妆模拟控制方法、装置、计算机设备及存储介质
CN110136698B (zh) * 2019-04-11 2021-09-24 北京百度网讯科技有限公司 用于确定嘴型的方法、装置、设备和存储介质
CN112308944A (zh) * 2019-07-29 2021-02-02 丽宝大数据股份有限公司 仿真唇妆的扩增实境显示方法
CN112017212B (zh) * 2020-08-26 2022-10-04 北京紫光展锐通信技术有限公司 人脸关键点跟踪模型的训练、跟踪方法及系统
CN112686820A (zh) * 2020-12-29 2021-04-20 北京旷视科技有限公司 虚拟美妆方法、装置和电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08202754A (ja) * 1995-01-27 1996-08-09 Pola Chem Ind Inc リップカラーアドバイスシステム及び方法
JP2007097950A (ja) * 2005-10-06 2007-04-19 Shiseido Co Ltd 唇の化粧方法
JP2009163465A (ja) * 2007-12-29 2009-07-23 Yurakusha:Kk 似顔絵イラストデータ提供システム
CN108804975A (zh) * 2017-04-27 2018-11-13 丽宝大数据股份有限公司 唇彩指引装置及方法
CN113674177A (zh) * 2021-08-25 2021-11-19 咪咕视讯科技有限公司 一种人像唇部的自动上妆方法、装置、设备和存储介质

Also Published As

Publication number Publication date
CN113674177A (zh) 2021-11-19
CN113674177B (zh) 2024-03-26

Similar Documents

Publication Publication Date Title
US20200043225A1 (en) Image processing apparatus and control method thereof
KR101554403B1 (ko) 화상 처리 장치, 화상 처리 방법, 및 제어 프로그램이 기록된 기억 매체
US9378564B2 (en) Methods for color correcting digital images and devices thereof
WO2019011147A1 (zh) 逆光场景的人脸区域处理方法和装置
JP2018088247A (ja) オブジェクトセグメンテーションのための前景マスク補正のための画像処理装置及び方法
TWI532385B (zh) 白平衡處理方法及其處理裝置
WO2015070723A1 (zh) 眼部图像处理方法和装置
US20160269708A1 (en) Image Processing Method and Apparatus
JP6458569B2 (ja) 画像処理装置および画像処理方法
CN106570838A (zh) 一种图像亮度优化方法及装置
US10055824B2 (en) Image correction device, image correction method and storage medium
GB2453890A (en) Pupil colour correction device and program
WO2018082388A1 (zh) 一种肤色检测方法、装置及终端
US10785466B2 (en) Multi-image color-refinement with application to disparity estimation
WO2023273246A1 (zh) 人脸图像处理方法及装置、计算机可读存储介质、终端
JP2007272435A (ja) 顔特徴抽出装置及び顔特徴抽出方法
JP5811416B2 (ja) 画像処理装置、画像処理方法及びプログラム
WO2023025239A1 (zh) 人像唇部的自动上妆方法、装置、设备、存储介质及程序产品
GB2453891A (en) Pupil color correction device and program
CN111627076B (zh) 换脸方法、装置及电子设备
WO2019129041A1 (zh) 一种亮度调节方法、装置、终端及计算机可读存储介质
JP7043992B2 (ja) 変更点検出装置、変更点検出方法及び変更点検出プログラム
JP2005316743A (ja) 画像処理方法および画像処理装置
US11240429B2 (en) Target object elimination method based on panoramic camera
CN113947708A (zh) 照明设备的灯效控制方法、系统、装置、电子设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22860585

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18573093

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE