WO2020177434A1 - 图像处理方法及装置、图像设备及存储介质 - Google Patents

图像处理方法及装置、图像设备及存储介质 Download PDF

Info

Publication number
WO2020177434A1
WO2020177434A1 PCT/CN2019/124515 CN2019124515W WO2020177434A1 WO 2020177434 A1 WO2020177434 A1 WO 2020177434A1 CN 2019124515 W CN2019124515 W CN 2019124515W WO 2020177434 A1 WO2020177434 A1 WO 2020177434A1
Authority
WO
WIPO (PCT)
Prior art keywords
face image
key points
apple muscle
face
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/124515
Other languages
English (en)
French (fr)
Inventor
苏柳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to JP2020535619A priority Critical patent/JP6990773B2/ja
Priority to KR1020207019432A priority patent/KR102386642B1/ko
Priority to SG11202006333VA priority patent/SG11202006333VA/en
Priority to US16/914,637 priority patent/US11238569B2/en
Publication of WO2020177434A1 publication Critical patent/WO2020177434A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/40—Filling planar surfaces by adding surface attributes, e.g. adding colours or textures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • This application relates to the field of computer technology, and in particular to an image processing method and device, image equipment and storage medium.
  • Mobile phones, tablets, or wearable devices are generally equipped with cameras, which can collect images.
  • the collected image may not be the ideal image that the user wants. It may be necessary to adjust the image so that the presented image meets the user's expectations, such as beautifying images, funny images, and cute images.
  • portrait processing as an example, related technologies
  • the image processing in is generally limited to the processing of eyes, nose and face, but the processing of other parts such as beautification is relatively small. Therefore, the image processing technology of the related technology is not optimized and perfect enough, and the image processing effect has not reached the desired expected effect.
  • embodiments of the present disclosure provide an image processing method and device, image equipment, and storage medium.
  • an image processing method including:
  • the target area is adjusted to form a second face image.
  • the method before the determining the target area of apple muscle adjustment based on the key points, the method further includes:
  • the determining the target area of apple muscle adjustment based on the key points includes:
  • the key points include: key points at the end of the eyes, key points on the first face contour, and key points on the nose wings;
  • the determining the target area for apple muscle adjustment based on the key point and the orientation includes:
  • the key points include key points at the end of the eyes, key points on the second face contour, and key points on the nose wings;
  • the determining the target area for apple muscle adjustment based on the key point and the orientation includes:
  • the target area is determined based on the second intermediate point and the key point at the end of the eye.
  • the determining the adjustment parameters of the apple muscle based on the key points includes:
  • the determining the adjustment direction of the apple muscle based on the key point and the orientation includes:
  • the adjustment direction of the apple muscle is determined to be the first direction based on the key points.
  • the determining the adjustment direction of the apple muscle based on the key point and the orientation includes:
  • the orientation indicates that the first face image is a side face image
  • the first direction is: the direction in which the first intermediate point of the target area points to the key point at the tail of the eye.
  • the second direction is: the direction in which the second intermediate point of the target area points to the key point at the tail of the eye;
  • the third direction is the direction in which the key point of the nose wing points to the key point of the second face contour in the target area.
  • the determining the adjustment parameters of the apple muscle based on the key points includes:
  • the adjustment range of the apple muscle is determined.
  • the determining the adjustment range of the apple muscle based on the key point and the orientation includes:
  • the actual movement distance of the apple muscle of the first face image is determined according to the face deflection angle and the maximum movement distance.
  • the method further includes:
  • the apple muscle in the second face image is brightened to obtain a third face image.
  • the brightening of the apple muscle in the second face image to obtain the third face image includes:
  • the brightening of the apple muscle in the second face image to obtain the third face image includes:
  • the generating a mask image according to the position of the apple muscle in the second face image includes:
  • the mask image including the bright spots is generated according to the preset bright spots of the apple muscle and the position of the apple muscle in the second face image.
  • the performing the apple muscle brightening process on the second face image based on the mask image to obtain the third face image includes:
  • the third face image is generated.
  • the method further includes:
  • the generating the third face image based on the second pixel value includes:
  • the third face image is generated.
  • an image processing device including:
  • the detection unit is used to detect the key points of the first face image
  • the first determining unit is configured to determine the target area of apple muscle adjustment based on the key points
  • the second determining unit is configured to determine the adjustment parameters of the apple muscle based on the key points
  • the adjustment unit is configured to adjust the target area to form a second face image based on the adjustment parameter.
  • the device further includes:
  • a third determining unit configured to determine the orientation of the face in the first face image
  • the first determining unit is used to:
  • the key points include: key points at the end of the eyes, key points on the first face contour, and key points on the nose wings;
  • the second determining unit is used for:
  • the key points include key points at the end of the eyes, key points on the second face contour, and key points on the nose wings;
  • the first determining unit is used to:
  • the target area is determined based on the second intermediate point and the key point at the end of the eye.
  • the second determining unit is configured to:
  • the third determining unit is configured to:
  • the adjustment direction of the apple muscle is the first direction based on the key points.
  • the third determining unit is further configured to:
  • the orientation indicates that the first face image is a side face image
  • the first direction is: the direction in which the first intermediate point of the target area points to the key point at the tail of the eye.
  • the second direction is: the direction in which the second intermediate point of the target area points to the key point at the tail of the eye;
  • the third direction is the direction in which the key point of the nose wing points to the key point of the second face contour in the target area.
  • the second determining unit is configured to:
  • the adjustment range of the apple muscle is determined.
  • the second determining unit is configured to:
  • the actual movement distance of the apple muscle of the first face image is determined according to the face deflection angle and the maximum movement distance.
  • the device further includes:
  • the brightening processing unit is configured to perform brightening processing on the apple muscle in the second face image to obtain a third face image.
  • the brightening processing unit is used to:
  • the brightening processing unit is used to:
  • the brightening processing unit is used to:
  • the mask image including the bright spots is generated according to the preset bright spots of the apple muscle and the position of the apple muscle in the second face image.
  • the brightening processing unit is used to:
  • the third face image is generated.
  • the device further includes:
  • the brightening processing unit is used for:
  • the third face image is generated.
  • an image device including:
  • the processor is connected to the memory and is configured to implement the first aspect and any one of its implementation modes by executing computer executable instructions stored on the memory.
  • a computer storage medium that stores computer-executable instructions; after the computer-executable instructions are executed, the first aspect and any one of its implementation modes can be implemented.
  • the technical solution provided by the embodiment of the present disclosure obtains the key points in the first face image by detecting the key points of the first face image, and then determines the target area where the apple muscle in the first face image is located according to the key points. And based on the key points, the adjustment parameters for adjusting the apple muscle of the first face image are determined, and the target area is adjusted by using the adjustment parameters to generate the second face image after the apple muscle adjustment.
  • the technical solution provided by the embodiments of the present disclosure provides at least the function of adjusting the apple muscle in the face image, and the apple muscle in the face image can be adjusted according to the user's needs subsequently, thereby improving the beautification and funnyness of the face image. Effect.
  • FIG. 1 is a schematic diagram of a pixel coordinate system provided by an embodiment of the disclosure
  • FIG. 2 is a schematic diagram of a region division of a face image provided by an embodiment of the disclosure
  • FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the disclosure.
  • FIG. 4 is a schematic diagram of key points of a human face provided by an embodiment of the disclosure.
  • FIG. 5 is a schematic flowchart of another image processing method provided by an embodiment of the disclosure.
  • 6A is a schematic diagram of the effect of a frontal portrait provided by an embodiment of the disclosure.
  • 6B is a schematic diagram of the effect of a profile portrait provided by an embodiment of the disclosure.
  • 6C is a schematic diagram of the effect of another profile portrait provided by an embodiment of the disclosure.
  • FIG. 7 is a schematic diagram of the effect of a mask image provided by an embodiment of the disclosure.
  • FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the disclosure.
  • FIG. 9 is a schematic structural diagram of an image device provided by an embodiment of the disclosure.
  • FIG. 10 is a schematic diagram of the effect of an image processing method provided by an embodiment of the disclosure.
  • the pixel coordinate system in the embodiment of the present disclosure.
  • Figure 1 take the lower right corner of the face image A as the origin o of the pixel coordinate system, the direction parallel to the row of the face image A as the x-axis direction, and the direction parallel to the column of the face image A as y
  • the direction of the axis constructs the pixel coordinate system xoy.
  • the abscissa is used to indicate the number of columns of pixels in the face image A in the face image A
  • the ordinate is used to indicate the number of rows of the pixels in the face image A in the face image A.
  • the units of the abscissa and the ordinate can both be pixels.
  • the coordinates of pixel a in Figure 1 are (10, 30), that is, the abscissa of pixel a is 10 pixels, the ordinate of pixel a is 30 pixels, and pixel a is the 10th column in face image A. Pixel in row 30.
  • the embodiment of the present disclosure divides the face area in the face image into a left face area and a right face area. As shown in Figure 2, the face area is divided into a left face area and a right face area based on the symmetrical center line of the face area.
  • this embodiment provides an image processing method, including:
  • Step S110 Detect key points of the first face image
  • Step S120 Determine the target area of the apple muscle adjustment based on the key points
  • Step S130 Determine the adjustment parameters of the apple muscle based on the key points
  • Step S140 Based on the adjustment parameters, adjust the target area to form a second face image.
  • the apple muscle in the face can be understood as the area where the inverted triangle-shaped tissue is located in the area under the eyes.
  • the facial muscles squeeze and bulge slightly.
  • the area where the inverted triangle-shaped tissue looks like a round and shiny apple is called apple muscle.
  • This embodiment provides an adjustment scheme for apple muscles, which can also be applied to other parts of the human face or the face parts of cartoon characters or the face parts of animals in practical applications.
  • the technical idea based on the adjustment scheme is for the above
  • the adjustment of facial parts can be regarded as within the technical scope protected by the embodiments of the present disclosure.
  • a deep learning model such as a neural network or a face key point detection algorithm is used to process the first face image to determine the face key points in the first face image.
  • the above-mentioned face key point detection algorithm can be OpenFace, multi-task cascaded convolutional networks (MTCNN), tweaked convolutional neural networks (TCNN), or task-constrained deep convolution A type of neural network (tasks-constrained deep convolutional network, TCDCN), the present disclosure does not limit the face key point detection algorithm.
  • the key points include but are not limited to the following key points:
  • Eyebrow key points for example, see key points 33 to 37, key points 38 to 42, and key point 64 to key point 71 in Figure 4;
  • the key points of the eye may include:
  • Key points on the inner corner of the eye for example, refer to key point 55 and key point 58 shown in Figure 4;
  • Key points at the outer corner of the eye for example, refer to key points 52 and 62 shown in FIG. 4;
  • Key points of the upper eyelid located on the upper eyelid for example, refer to key points 53, 72, and 54, and key points 59, 75, and 60 shown in Figure 4;
  • the key points of the lower eyelid located on the lower eyelid for example, see key points 62, 76 and 63 shown in Figure 4; key points 57, 73 and 56;
  • the key points of the eyeball located on the eyeball for example, see the key points 74 and 104 and the key points 77 and 105 shown in FIG. 4;
  • the key points of the nose can be divided into:
  • the key points of the wing of the nose located on the wing of the nose for example, refer to the key points 80 to 83 of Fig. 4;
  • the key points on the bottom of the nose are located at the bottom of the nose; for example, refer to key points 47 to 51 shown in FIG. 4.
  • the key points related to apple muscle adjustment include at least one of the following:
  • step S130 first, according to one or more of the key points, a target area for apple muscle adjustment is circled in the first face image. After delimiting the area, the apple muscle can be adjusted according to the pixels in the area.
  • the first facial contour key point and/or the second facial contour key point can be selected from the above facial contour key points, so as to determine the adjustment parameters of the apple muscle.
  • the target area may be a circular area or an elliptical area.
  • step S130 the adjustment parameters of the apple muscle are determined according to one or more of the key points.
  • the adjustment parameters of the apple muscle include but are not limited to at least one of the following:
  • the adjustment direction can be used to determine the moving direction of the apple muscle on the first face image
  • the adjustment range can be used to determine the change scale of the apple muscle on the first face image, for example, the moving distance or zoom ratio;
  • the adjustment gradient can be used to adjust the difference in the amplitude of different times during multiple adjustments, or the difference in the adjustment amplitude between different areas when performing sub-regional adjustment of the apple muscle. For example, divide a circular or oval target area into an inner area and an outer area; the outer area surrounds the inner area. When adjusting the brightness of the apple muscle, the brightness adjustment range of the inner area is greater than that of the outer area. The difference between the adjusted brightness of the inner and outer regions constitutes the adjustment gradient. Of course, this is only an example, and there are many specific implementation methods.
  • the key points used to determine the adjustment parameter and the target area may be the same or different. In some embodiments, the key points used to determine the adjustment parameter and the target area may be partially the same. For example, the same facial contour key points may be used to determine the adjustment parameter and the target area. For another example, the same key points of the eye may be used at the same time to determine the adjustment parameter and the target area.
  • step S140 at least one of the position, shape, size, or color of the target area in the first face image is adjusted according to the adjustment parameters to form a second face image.
  • the step S140 may include at least one of the following:
  • the shape of the target area is changed, and the shape of the target area is adjusted, so that the shape of the apple muscle that matches the shape of the human face is imaged.
  • different portraits image different face shapes, for example, some are oval-shaped faces, some are heart-shaped faces, some are round faces, and some are prismatic faces.
  • the general shape of the apple muscle is an inverted triangle; however, the angle of the three corners of the inverted triangle or the position of the vertex on the face image are different, which will make the apple muscle present different visual feelings. Therefore, in some embodiments, the imaging device can also adjust the position of the pixels in the target area, etc., so that the apple muscle presents inverted triangles of different shapes in the target area or in the target area after movement, so as to fit the face shape. Matching, so as to achieve portrait beautification, or comical treatment.
  • a method is provided to adjust the apple muscle in the face image, so as to realize the beautification, funny, cuteness or different styles of apple muscle in the face image; so, The apple muscle processing of the image can be realized according to the user's needs, so as to meet the user's different image requirements, and improve the user experience and image processing quality.
  • the adjustment parameters when generating the adjustment parameters in step S130, may be generated according to the current expected adjustment effect of the apple muscle.
  • the currently expected adjustment effect is an apple skin beautification effect
  • the apple muscle can present a popular aesthetic beautification effect on the second face image relative to the first face image according to the beautification effect.
  • the current expected adjustment effect is a comic effect, so as to present a humorous effect through the adjustment of the apple muscle on the first face image; then when generating the adjustment parameters, the key points and the The comic effect generates apple muscle that makes the second face image present a humorous and funny effect.
  • the generation of the adjustment parameter is also determined according to the expected adjustment effect.
  • the adjustment of the apple muscle can be used in the image beautification process.
  • the apple muscle adjustment provided by the embodiment of the present disclosure can be used in one-click portrait beauty pictures. Skin beautification, beautify the eyes, nose or face as a function of one-click portrait beauty.
  • the adjustment of the apple muscle provided by this embodiment can also be used in the special apple muscle beautification function.
  • an apple skin beautification control is provided in the Meitu application, and if an operation of the apple skin beautification control is detected by the user, the apple skin in the image is separately beautified using the above method.
  • the image processing method described in this embodiment has the function of adjusting the apple muscle, and the apple muscle can be adjusted in the process of portrait processing, so as to meet the user's demand for apple muscle adjustment and improve the user experience and image equipment.
  • the intelligence is the intelligence.
  • the method includes:
  • Step S111 Determine the orientation of the face in the first face image
  • the step S120 may include step S121; the step S121 may include: determining the target area for apple muscle adjustment based on the key points and the orientation.
  • the orientation of the first face image is also determined.
  • the orientation of the first face image is divided into at least two types:
  • front face orientation means that the first face image is a front face image
  • side face orientation means that the first face image is a side face image.
  • the orientation of the first face image is the frontal orientation.
  • the aforementioned face deflection angle refers to the angle between the shooting direction of the imaging device and the vertical line of the face area of the person being photographed, and when viewed from the top of the person’s head, the shooting direction of the imaging device is compared When the offset direction of the symmetrical center line of the photographed person’s face area is clockwise, the face deflection angle is positive. On the contrary, when viewed from the top of the photographed person’s head, the imaging device’s shooting direction is relatively When the offset direction of the symmetrical center line of the face region of the person being photographed is counterclockwise, the face deflection angle is negative.
  • the orientation of the first face image is the side face orientation.
  • the first preset range and the second preset range are complementary to each other.
  • the first preset range is -10 degrees to 10 degrees
  • the second preset range is -180 degrees to -10 degrees, and 10 degrees to 180 degrees.
  • all orientations other than the front face orientation can be regarded as the side face orientation, except for the back image that does not contain the human face at this time.
  • Fig. 6A is a face image with a front face
  • Fig. 6B is a face image with a side face
  • Fig. 6C is another face image with a side face.
  • the face deflection angle of the face image shown in FIG. 6A is 0 degrees, assuming that the face deflection angle of the face image shown in FIG. 6B is -90 degrees, the face image shown in FIG. 6C The deflection angle of the face is 90 degrees.
  • the first preset range and the second preset range can be set as required.
  • the area covered by the apple muscle in the face image with the front face (hereinafter referred to as the front face image) and the area covered by the apple muscle in the face image with the side face (hereinafter referred to as the side face image) Different (including the area covered by the apple muscle, the angle of the area covered by the apple muscle, etc.), if the same adjustment standard or adjustment parameter is used for the front face image and the side face image, it will cause the difference in the face image
  • the adjustment effect of the apple muscle is not good (for example: the ratio of the adjusted apple muscle area to the area of the face area is not appropriate, another example: the color of the adjusted apple muscle area and the non-apple muscle area in the face area The color difference is large).
  • the embodiments of the present disclosure adopt different methods to determine adjustment parameters for face images of different orientations, so as to improve the effect of adjusting the apple muscle of the face image.
  • the step S120 may include:
  • the first intermediate point of the target area is determined based on the first facial contour key points and the nose key points.
  • the first facial contour key point may be a facial contour key point whose ordinate is within a first target range, where the first target range is the sum of the ordinate range of the area covered by the apple muscle and the third preset range.
  • the above-mentioned third preset range is -5 pixels to 5 pixels.
  • key point 4 and key point 5 can be used as the first facial contour key points for adjusting the left apple muscle (that is, the apple muscle located in the left face area) in the face image.
  • the key point 5 is preferably used as the first face contour key point.
  • the key point of the nose wing is a key point where the difference between the ordinate and the ordinate of the first face contour key point is within the fourth preset range and is located in the wing area (the area can be defined according to user requirements). For example, suppose that the maximum value of the ordinate of the key point in the first face contour key point is 50 pixels, the minimum value is 40 pixels, and the fourth preset range is 10 pixels, then the wing key point is that the ordinate is greater than Nose wing key points within the range of 30 pixels or less and 60 pixels or less.
  • the aforementioned key point of the nose wing may be the key point 80 or the key point 82 shown in FIG. 4.
  • the key point 80 is preferably selected as the key point of the nose wing.
  • the key points and key points 80 shown in FIG. 4 are selected as the first facial contour for adjusting the right apple muscle (that is, the apple muscle located in the right face area).
  • the key points and the key points of the nose the adjustment effect more meets the user's expectations.
  • the key points 27 and the key points 81 shown in FIG. 4 are selected as the first facial contour key points and the nose key points of the apple muscle of the left face, respectively.
  • the first intermediate point includes but is not limited to: the midpoint of the first facial contour key point and the nose key point (the midpoint between the key point 80 and the key point 82 shown in FIG. 4, the key point 81 and The midpoint between key points 83).
  • the face image presented by the first face image has different facial features
  • the first intermediate point is obtained after position correction is performed based on the midpoint of the first facial contour key point and the nose key point. For example, if the first face image presents face A, it is assumed that the distance between the key point of the eyeball and the key point of the first face contour at the corresponding position is the first distance, and the key point of the eyeball is assumed to be the key point of the nose at the corresponding position The distance of the midpoint of the point is the second distance.
  • the first face contour key point at the above corresponding position refers to: if the eye key point is located in the left face area of the face area, the first face contour key point corresponding to the eye key point is the first face key point located in the left face area The first face contour key point closest to the eyeball key point; if the eyeball key point is located in the right face area of the face area, the first face contour key point corresponding to the eyeball key point is the first face contour key point located in the right face area Among the key points, the first face contour key point closest to the eyeball key point.
  • the midpoint of the nose key points at the corresponding positions above refers to: if the eye key points are located in the left face area of the face area, the mid point of the nose key points corresponding to the eye key points is the midpoint of the nose key points located in the left face area; If the eyeball key point is located in the right face area of the human face area, the midpoint of the nose key point corresponding to the eyeball key point is the midpoint of the nose key point located in the right face area.
  • the correction parameter can be generated according to the ratio. If the ratio of the first distance to the second distance is 1, the midpoint of the first facial contour key point corresponding to the eyeball key point and the nose key point corresponding to the eyeball key point is taken as the first intermediate point.
  • first determine the midpoint of the line between the first facial contour key point corresponding to the eye key point and the nose key point corresponding to the eye key point (This will be referred to as the first intermediate point to be confirmed hereinafter) is the amount of shift to the symmetrical centerline or to the contour of the face, and the shift includes a lateral shift and a longitudinal shift. The point determined by the sum of the coordinates of the first intermediate point to be confirmed and the offset is taken as the first intermediate point.
  • the intermediate point to be confirmed can be moved toward the face contour line by the offset, for example, if the key points of the eyeball and the first face contour are both If it is located in the left face area, the horizontal offset can be taken as a negative value to make the first intermediate point to be confirmed move to the face contour line; if the eyeball key points and the first face contour key points are both located in the left face area, then The lateral offset can be taken as a positive value to move the first intermediate point to be confirmed to the symmetrical centerline; if the eyeball key points and the first face contour key points are located in the right face area, the lateral offset can be taken Is a positive value to make the first intermediate point to be confirmed move toward the face contour; if the eyeball key point and the first face contour key point are both located in the right face area, the horizontal offset can be taken as a negative value to make The first intermediate point to be confirmed moves to the center line of symmetry.
  • the target area can be determined in the face area according to the radius of the target area set by the user and the first intermediate point.
  • the radius of the target area may be determined according to the area of the face area.
  • the radius of the target area may also be estimated by the deep learning model based on the first face image.
  • step S120 further includes:
  • the linear interpolation algorithm is used to calculate the circle radius or the long axis and the short axis of the target area with the coordinates of the first intermediate point and the key point at the end of the eye in the first face image as known quantities. axis. If the target area is a circle, the first intermediate point is the center of the circular area, and the calculated radius of the circle of the target area is the radius, the target area can be constructed.
  • the first intermediate point is the center of the elliptical area
  • one of the major axis or minor axis is determined, and then based on the preset ratio between the major axis and the minor axis, the other axis is also determined If the center, major axis and minor axis are determined, the natural elliptical target area is also determined.
  • the step S120 may include:
  • the second intermediate point of the target area is determined based on the second facial contour key points and the nose key points.
  • the second facial contour key point in the side face image is different from the first facial contour key point of the front face image.
  • the second face contour key point may be a face contour key point whose ordinate is within the second target range, where the second target range is the sum of the ordinate range of the area covered by the apple muscle and the fifth preset range. Set the range to be smaller than the third preset range.
  • the key points 4, 5, 27 and 28 shown in FIG. 4 are all key points of the second face contour.
  • the key points of the nose wing may be the same as the key points of the nose wing of the front face image, and the key point 80 or the key point 81 shown in FIG. 4 can be selected for reference.
  • the second intermediate point can also be the midpoint between the second facial contour key point and the nose key point (hereinafter referred to as the second midpoint to be confirmed), or the offset is used in the reference frontal image. Moving the first intermediate point to be confirmed to obtain the first intermediate point by moving the second intermediate point to be confirmed in the side face image to obtain the second intermediate point.
  • the second intermediate point may be the center point of the target area.
  • step S120 may further include: determining the range of the target area based on the second intermediate point and the key point at the tail of the eye.
  • the second intermediate point and the key point at the end of the eye can also be based on a linear interpolation algorithm to obtain parameters of a certain range such as the radius of the target area, so as to determine the parameter on the first face image.
  • a linear interpolation algorithm to obtain parameters of a certain range such as the radius of the target area, so as to determine the parameter on the first face image.
  • the scope of the target area is not limited to the range of the target area.
  • the step S130 may include:
  • the adjustment parameters of the apple muscle are determined.
  • Different orientations may have different adjustment strategies for apple muscles. For example, the adjustment direction and/or angle of the adjustment of the front face image and the adjustment of the side face image are different, and the corresponding adjustment strategies are also different. For another example, if the orientation is different, the brightness adjustment of the apple muscle is different and/or the adjustment gradient is different.
  • the step S130 may specifically include: determining the adjustment direction of the apple muscle based on the key point and the orientation.
  • the adjustment direction of the apple muscle is different if the orientation is different.
  • the determining the adjustment direction of the apple muscle based on the key point and the orientation includes:
  • the orientation indicates that the first face image is a side face image
  • the third direction is different from the second direction, including but not limited to: the third direction is perpendicular to the second direction.
  • the target area will be moved in two directions, one is to pull the target area up toward the eye, and the other is to move the target area out of the edge of the face, optional, third
  • the direction is perpendicular to the second direction.
  • the first direction may be the direction in which the first intermediate point points to the key point of the tail of the eye;
  • the second direction may be the second intermediate point Point to the direction of the key point at the end of the eye;
  • the third direction may be: the second facial contour key point points to the opposite direction of the nose key point (ie, from the nose key point to the second facial contour The direction of the key point).
  • the distribution itself reflects the characteristics of the face. Therefore, the adjusted first point is determined based on the face contour key points and the eye tail key points.
  • the one direction, the second direction and the third direction can essentially adjust the personality of different faces to realize the optimization and adjustment of the apple muscle.
  • the step S130 may include:
  • the maximum moving distance of the apple muscle is determined.
  • the distance between the key point at the end of the eye and the key point at the head of the eye is essentially the length of the eye in the first face image; for example, the key point at the tail of the eye and the key point at the head of the eye are separated by M pixels; then the maximum The distance can be M*A pixels.
  • A can be a positive number less than 1. Further, for example, the value of A may be a positive number between 0.44 and 0.74.
  • the actual movement distance of the apple muscle may be determined based on user operations. For example, the user inputs a user operation on a human-computer interaction interface, and the actual movement distance is determined according to the user operation. If the movement distance calculated based on the user operation is greater than the maximum movement distance, the apple muscle is adjusted by using the maximum movement distance as the actual movement distance of the apple muscle.
  • the step S130 may include:
  • the actual moving distance of the apple muscle of the side face image is determined according to the face deflection angle and the maximum moving distance.
  • the maximum movement distance of the apple muscle can be determined based on the key points, and the maximum movement distance is also the maximum movement distance of the target area.
  • the key points of the eyes are used as the key points, and the maximum moving distance of the apple muscle is determined based on the size of the eyes.
  • the actual movement distance described above is positively correlated with the deflection angle of the face in the side face image.
  • the actual movement distance may be the above-mentioned maximum movement distance.
  • the actual moving distance the above-mentioned maximum moving distance*a, where a is a positive number less than 1.
  • the method further includes:
  • Step S150 Perform brightening processing on the apple muscle in the second face image to obtain a third face image.
  • the second face image relative to the first face image makes the apple muscle move on the face.
  • the moved apple muscle can also be brightened, so that the apple muscle looks more full and full.
  • pixels at different positions in the apple muscle have different brightening degrees.
  • the brightness of the highest point of the apple muscle is gradually reduced to the periphery of the apple muscle, so that the brightness of the highest point of the apple muscle is maximized, and then gradually decreases to the same brightness as the surrounding skin pixels of the apple muscle. It makes the apple muscle look more natural after adjustment, and uses the brightness difference of different positions of the apple muscle to reflect the three-dimensional effect of the apple muscle.
  • the method further includes:
  • the step S150 may include: performing a brightening process on the apple muscle in the second face image based on the brightening parameter to obtain the third face image.
  • the skin color parameter of the face image is acquired, and the skin color parameter includes but is not limited to at least one of the following:
  • the color value of the skin pixel in the face image and the color value of the skin pixel between the eyes and lips in the face image, the gray histogram of the skin pixels in the face image, and the difference between the eyes and lips in the face image The color value of the skin pixel, the color value of the skin pixel in the face image, the gray histogram of the skin pixel between the eyes and the lips in the face image, the gray histogram of the skin pixel in the face image, and the face image Grayscale histogram of skin pixels between middle eyes and lips.
  • the average brightness or median brightness of the apple muscle and the skin around the apple muscle can be determined, and then the brightening parameters of the pixels in the apple muscle area can be determined based on the average or median brightness to distinguish the apple muscle from
  • the skin around the apple muscle can also reduce the difference between the apple muscle and the skin around the apple muscle.
  • the brightening parameter includes but is not limited to at least one of the following: brightening amplitude, and brightening ratio.
  • a mask image is used for brightening.
  • the apple muscle brightening may include:
  • the brightening range Based on the brightening range, you can add a corresponding brightening range to the original pixel value of the apple muscle to get the brightened pixel value. After the brightening processing of all pixels in the apple muscle area is completed, the brightening can be obtained After the apple muscle, and obtain the third face image;
  • the apple muscle brightening may include:
  • the original pixel value of the apple muscle can be multiplied by the brightening ratio to obtain the brightened pixel value.
  • the brightened pixel value can be obtained Apple muscle and obtain the third face image.
  • the step S150 may include: generating a mask image according to the position of the apple muscle in the second face image; and performing an analysis on the second face image based on the mask image. The brightening processing of the apple muscle obtains the third face image.
  • the step S150 may include:
  • a mask image of the same size as the second face image is formed based on the location; the mask image has a semi-transparent brightening halo at the location of the second face image; other locations are transparent areas ; Superimpose the mask image on the second face image to generate the third face image.
  • the brightness value of the brightening halo in the mask image can directly be the brightness value after the apple muscle is brightened, so the brightening parameter may also include: the brightness value after the apple muscle is brightened.
  • FIG. 7 shows a schematic diagram of providing a mask image in this embodiment.
  • the white halo area is used to overlap the adjusted apple muscle area to brighten the apple muscle.
  • Another way to brighten the apple muscle is: generating a mask image according to the position of the apple muscle in the second face image, including:
  • the performing the apple muscle brightening processing on the second face image based on the mask image to obtain the third face image includes:
  • the third face image is generated.
  • the bright spot of the apple muscle may be predetermined, and the shape and brightness of the bright spot may be predetermined. In this way, the generation rate of the mask image can be accelerated.
  • the position of the bright spot of the apple muscle in the mask image corresponds to the position of the apple muscle in the second face image.
  • the area of the apple muscle in the second face image can be determined according to the key points detected in step S110; for example, the target area determined in step S120; or, based on the target area and the actual moving distance, The area where the apple muscle in the second face image is located is added to the mask image corresponding to the bright spot based on the area.
  • the pixel value of the mask image except for the position of the bright spot will be greater than the predetermined threshold, and the pixel value of the pixel except the position of the bright spot will be less than or equal to the predetermined threshold.
  • an arithmetic module suitable for a large number of calculations such as an image processor (GPU) can be used.
  • the pixel value is compared with a predetermined threshold to determine whether the pixel needs to be compared with
  • the pixels in the second face image are mixed. For example, if the second face image is an RGB image, the second face image includes 3 color channels, and each pixel in these 3 color channels has a corresponding pixel value.
  • the Mth pixel in the mask image is If the pixel value is greater than the predetermined threshold, then the pixel value of the Mth pixel in the mask image and the color values of the three color channels of the Mth pixel in the second face image are mixed, and then these 3 colors are combined
  • the mixed value of the channels obtains the second pixel value.
  • the mixing here may be linear mixing or nonlinear mixing.
  • the blending of the mask image and the second face image is preferably a non-linear blending, and the resulting apple muscle has a better brightening effect.
  • Sqrt represents the square root
  • Sqrt (original color) represents the square root of the first pixel value
  • the value of a and the value of b may be the same, for example, after the original color and mask color are normalized to a value between 0 and 1, the values of a and b Both can be 2.0; the value of c can be 1.0.
  • a, b, and c may be the same or different.
  • the pixel value or color value is normalized before the pixel value mixing, and the obtained pixel value or color value is data between 0 and 1.
  • the normalization process greatly reduces the number of subsequent pixel mixing calculations, thereby simplifying the calculation.
  • the predetermined threshold may be 0.4, 0.45, 0.5, 0.55, 0.6, etc.
  • the value range of the predetermined threshold may be between 0.4 and 0.6.
  • the method further includes: obtaining control parameters;
  • the generating the third face image based on the second pixel value includes:
  • the third face image is generated.
  • control parameter can be an external control parameter received by the image device from the human-computer interaction interface or from other devices, and can be used to control the brightness level of the apple muscle. It is a control parameter for the brightness level of the apple muscle. .
  • A may be the control parameter;
  • the RGB1 may be the original first pixel value of the second face image;
  • RGB2 may be the second pixel value after mixing;
  • RGB3 may be the second pixel value after mixing again;
  • the second pixel value is not directly used to generate the third face image, but the original second face image and the mixed second pixel value are used for linear mixing again to obtain the third face image. Face image, in this way, can make the brightening of the apple muscle more natural, and the image processing effect is more superior.
  • the embodiments of the present disclosure describe in detail how to adjust the target area of the apple muscle in the face image, and in practical applications, the forehead area and the forehead area in the face image can also be adjusted according to the technical solutions provided by the embodiments of the present disclosure. Adjust the jaw area and cheekbone area.
  • this embodiment provides an image processing apparatus 1, including:
  • the detection unit 11 is used to detect the key points of the first face image
  • the first determining unit 12 is configured to determine the target area for apple muscle adjustment based on the key points;
  • the second determining unit 13 is configured to determine the adjustment parameters of the apple muscle based on the key points;
  • the adjustment unit 14 is configured to adjust the target area based on the adjustment parameters to form a second face image.
  • the device 1 further includes:
  • the third determining unit 15 is configured to determine the orientation of the face in the first face image
  • the first determining unit 12 is used to:
  • the key points include: key points at the end of the eyes, key points on the first face contour, and key points on the nose wings;
  • the second determining unit 13 is used to:
  • the key points include key points at the end of the eyes, key points on the second face contour, and key points on the nose wings;
  • the first determining unit 12 is used to:
  • the target area is determined based on the second intermediate point and the key point at the end of the eye.
  • the second determining unit 13 is configured to:
  • the third determining unit 15 is configured to:
  • the adjustment direction of the apple muscle is the first direction based on the key points.
  • the third determining unit 15 is further configured to:
  • the orientation indicates that the first face image is a side face image
  • the first direction is: the direction in which the first intermediate point of the target area points to the key point at the tail of the eye.
  • the second direction is: the direction in which the second intermediate point of the target area points to the key point at the tail of the eye;
  • the third direction is the direction in which the key point of the nose wing points to the key point of the second face contour in the target area.
  • the second determining unit 13 is configured to:
  • the adjustment range of the apple muscle is determined.
  • the second determining unit 13 is configured to:
  • the actual movement distance of the apple muscle of the first face image is determined according to the face deflection angle and the maximum movement distance.
  • the device 1 further includes:
  • the brightening processing unit 16 is configured to brighten the apple muscle in the second face image to obtain a third face image.
  • the brightening processing unit 16 is configured to:
  • the brightening processing unit 16 is configured to:
  • the brightening processing unit 16 is configured to:
  • the mask image including the bright spots is generated according to the preset bright spots of the apple muscle and the position of the apple muscle in the second face image.
  • the brightening processing unit 16 is configured to:
  • the third face image is generated.
  • the device 1 further includes:
  • the obtaining unit 17 is used to obtain control parameters
  • the brightening processing unit 16 is used for:
  • the third face image is generated.
  • Apple muscle adjustment method can include the following steps:
  • the apple muscle is pulled up or expanded laterally on the area where the apple muscle of the first face image is located to obtain the second face image.
  • the dotted circular area of the second face image relative to the first face image can be the target area where the apple muscle is located; the second mask image and the second face image are combined to obtain the apple muscle Brightened third face image.
  • an example of apple muscle adjustment including:
  • the area of the apple muscle range of the first face image (corresponding to the above target area) is pulled upward;
  • the upward pulling direction is the direction in which the middle point of the pulling area points to the key point 52 (target point), and the range (radius) of the pulling area is obtained by linear interpolation from the middle point to the target point;
  • the first face image is a side face image
  • this area is expanded outward according to the face deflection angle of the first face image
  • an expansion area can be determined.
  • the expansion area takes key point 4 as the middle point.
  • the maximum movement distance of the expansion area is the same as the lifting distance and is positively correlated with the face deflection angle.
  • the expansion area takes key point 4 as the middle point, and the expansion direction is the opposite direction from key point 4 to key point 80.
  • the range (radius) of the expansion area is obtained by linear interpolation from the middle point to the target point;
  • the brightening process is performed, using a mask texture, which highlights the standard facial apple muscle area, and then blends the colors with the first face image, so that the facial apple muscle area will be brightened. It looks more three-dimensional.
  • this embodiment provides an image device, including:
  • the processor is connected to the memory, and is configured to implement the image processing method provided by any of the foregoing technical solutions by executing computer-executable instructions located on the memory, for example, the image processing method shown in FIG. 1 and/or FIG. 3 .
  • the memory can be various types of memory, such as random access memory, read-only memory, flash memory, etc.
  • the memory can be used for information storage, for example, storing computer executable instructions and the like.
  • the computer-executable instructions may be various program instructions, for example, target program instructions and/or source program instructions.
  • the processor may be various types of processors, for example, a central processing unit, a microprocessor, a digital signal processor, a programmable array, a digital signal processor, an application specific integrated circuit, or an image processor.
  • the processor may be connected to the memory through a bus.
  • the bus may be an integrated circuit bus or the like.
  • the image device may further include: a communication interface
  • the communication interface may include: a network interface, for example, a local area network interface, a transceiver antenna, and the like.
  • the communication interface is also connected to the processor and can be used for information transmission and reception.
  • the image device further includes a human-computer interaction interface.
  • the human-computer interaction interface may include various input and output devices, such as a keyboard and a touch screen.
  • This embodiment provides a computer storage medium that stores computer executable instructions; after the computer executable instructions are executed, they can be applied to one or more technologies in image equipment, databases, and the first private network
  • the image processing method provided by the solution for example, the image processing method shown in FIG. 1 and/or FIG. 2.
  • the computer storage medium may include various recording media with recording functions, for example, various storage media such as CD, floppy disk, hard disk, magnetic tape, optical disk, U disk, or mobile hard disk.
  • the optional computer storage medium may be a non-transitory storage medium, which can be read by the processor, so that after the computer executable instructions stored on the computer storage mechanism are acquired and executed by the first processor,
  • the image processing method provided by any one of the foregoing technical solutions can be implemented, for example, the image processing method applied in an image device or an image processing method in an application server can be executed.
  • This embodiment also provides a computer program product that includes computer-executable instructions; after the computer-executable instructions are executed, the image processing method provided by one or more technical solutions can be implemented, for example, , The image processing method shown in Figure 3 and/or Figure 5.
  • the computer program includes a computer program tangibly contained on a computer storage medium, and the computer program includes program code for executing the method shown in the flowchart.
  • the program code may include instructions corresponding to the steps of the method provided in the embodiments of the present invention.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, such as: multiple units or components can be combined, or It can be integrated into another system, or some features can be ignored or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms of.
  • the units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the functional units in the embodiments of the present disclosure can be all integrated into one processing module, or each unit can be individually used as a unit, or two or more units can be integrated into one unit;
  • the unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.
  • the foregoing program can be stored in a computer readable storage medium, and when the program is executed, it is executed Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.
  • ROM read-only memory
  • RAM random access memory
  • magnetic disks or optical disks etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种图像处理方法及装置、图像设备及存储介质。所述方法包括：检测第一人脸图像的关键点（S110）；基于所述关键点，确定苹果肌调整的目标区域（S120）；基于所述关键点，确定苹果肌的调整参数（S130）；基于所述调整参数，调整所述目标区域形成第二人脸图像（S140）。

Description

图像处理方法及装置、图像设备及存储介质
本申请要求于2019年03月06日提交中国专利局、申请号为201910169460.X、发明名称为“图像处理方法及装置、图像设备及存储介质”，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种图像处理方法及装置、图像设备及存储介质。
背景技术
手机、平板电脑或可穿戴设备等一般都配置了摄像头，摄像头可以采集图像。但是采集的图像未必是用户想要的理想图像，可能需要对图像进行调整，使得呈现的图像符合用户的期望，例如，美化图像、滑稽化图像、可爱化图像，以人像处理为例，相关技术中对于图像的处理，一般局限于眼睛、鼻子和脸型的处理，但是对于其他部分的美化等处理相对较少。故相关技术的图像处理技术还不够优化和完善，图像的处理效果还达不到理想的预期效果。
发明内容
有鉴于此，本公开实施例提供了一种图像处理方法及装置、图像设备及存储介质。
第一方面，提供了一种图像处理方法，所述方法包括：
检测第一人脸图像的关键点；
基于所述关键点，确定苹果肌调整的目标区域；
基于所述关键点，确定苹果肌的调整参数；
基于所述调整参数，调整所述目标区域形成第二人脸图像。
结合本公开任一实施方式，在所述基于所述关键点，确定苹果肌调整的目标区域之前，所述方法还包括：
确定所述第一人脸图像中人脸的朝向；
所述基于所述关键点，确定苹果肌调整的目标区域，包括：
基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
结合本公开任一实施方式，所述关键点包括：眼尾关键点、第一脸型轮廓关键点和鼻翼关键点；
所述基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域，包括：
在确定所述朝向表示第一人脸图像为正脸图像的情况下，基于所述第一脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第一中间点；
基于所述第一中间点及所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述关键点包括眼尾关键点、第二脸型轮廓关键点和鼻翼关键点；
所述基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域，包括：
在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于所述第二脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第二中间点；
基于所述第二中间点与所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述基于所述关键点，确定苹果肌的调整参数，包括：
基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
结合本公开任一实施方式，所述基于所述关键点及所述朝向，确定所述苹果肌的调整方向，包括：
在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调 整方向为第一方向。
结合本公开任一实施方式，所述基于所述关键点及所述朝向，确定所述苹果肌的调整方向，包括：
在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
结合本公开任一实施方式，所述第一方向为：所述目标区域的第一中间点指向眼尾关键点的方向。
结合本公开任一实施方式，所述第二方向为：所述目标区域的第二中间点指向眼尾关键点的方向；
所述第三方向为：鼻翼关键点指向确定目标区域的第二脸型轮廓关键点的方向。
结合本公开任一实施方式，所述基于所述关键点，确定苹果肌的调整参数，包括：
基于所述关键点和所述朝向，确定所述苹果肌的调整幅度。
结合本公开任一实施方式，所述基于所述关键点和所述朝向，确定所述苹果肌的调整幅度，包括：
基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离；
在所述朝向表示所述第一人脸图像为侧脸图像的情况下，根据人脸偏转角及所述最大移动距离，确定所述第一人脸图像的苹果肌的实际移动距离。
结合本公开任一实施方式，所述方法还包括：
对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
结合本公开任一实施方式，所述对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像，包括：
从所述第二人脸图像中获取人脸的肤色参数；
基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，获得所述第三人脸图像。
结合本公开任一实施方式，所述对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像，包括：
根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；
基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
结合本公开任一实施方式，所述根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像，包括：
根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的所述掩码图像。
结合本公开任一实施方式，所述基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像，包括：
将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
基于所述第二像素值，生成所述第三人脸图像。
结合本公开任一实施方式，所述方法还包括：
获取控制参数；
所述基于所述第二像素值，生成所述第三人脸图像，包括：
基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素值；
基于所述第三像素值，生成所述第三人脸图像。
第二方面，提供了一种图像处理装置，包括：
检测单元，用于检测第一人脸图像的关键点；
第一确定单元，用于基于所述关键点，确定苹果肌调整的目标区域；
第二确定单元，用于基于所述关键点，确定苹果肌的调整参数；
调整单元，用于基于所述调整参数，调整所述目标区域形成第二人脸图像。
结合本公开任一实施方式，所述装置还包括：
第三确定单元，用于确定所述第一人脸图像中人脸的朝向；
所述第一确定单元用于：
基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
结合本公开任一实施方式，所述关键点包括：眼尾关键点、第一脸型轮廓关键点和鼻翼关键点；
所述第二确定单元用于：
在确定所述朝向表示第一人脸图像为正脸图像的情况下，基于所述第一脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第一中间点；
基于所述第一中间点及所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述关键点包括眼尾关键点、第二脸型轮廓关键点和鼻翼关键点；
所述第一确定单元用于：
在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于所述第二脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第二中间点；
基于所述第二中间点与所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述第二确定单元用于：
基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
结合本公开任一实施方式，所述第三确定单元用于：
在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第一方向。
结合本公开任一实施方式，所述第三确定单元还用于：
在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
结合本公开任一实施方式，所述第一方向为：所述目标区域的第一中间点指向眼尾关键点的方向。
结合本公开任一实施方式，所述第二方向为：所述目标区域的第二中间点指向眼尾关键点的方向；
所述第三方向为：鼻翼关键点指向确定目标区域的第二脸型轮廓关键点的方向。
结合本公开任一实施方式，所述第二确定单元用于：
基于所述关键点和所述朝向，确定所述苹果肌的调整幅度。
结合本公开任一实施方式，所述第二确定单元用于：
基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离；
在所述朝向表示所述第一人脸图像为侧脸图像的情况下，根据人脸偏转角及所述最大移动距离，确定所述第一人脸图像的苹果肌的实际移动距离。
结合本公开任一实施方式，所述装置还包括：
提亮处理单元，用于对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
结合本公开任一实施方式，所述提亮处理单元用于：
从所述第二人脸图像中获取人脸的肤色参数；
基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，获得所述第三人脸 图像。
结合本公开任一实施方式，所述提亮处理单元用于：
根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；
基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
结合本公开任一实施方式，所述提亮处理单元用于：
根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的所述掩码图像。
结合本公开任一实施方式，所述提亮处理单元用于：
将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
基于所述第二像素值，生成所述第三人脸图像。
结合本公开任一实施方式，所述装置还包括：
获取单元，用于获取控制参数；
所述提亮处理单元用于：
基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素值；
基于所述第三像素值，生成所述第三人脸图像。
第三方面，提供了一种图像设备，包括：
存储器；
处理器，与所述存储器连接，用于通过执行存储在所述存储器上的计算机可执行指令，实现第一方面及其任意一种实施方式。
一种计算机存储介质，所述计算机存储介质存储有计算机可执行指令；所述计算机可执行指令被执行后，能够实现前第一方面及其任意一种实施方式。
本公开实施例提供的技术方案，通过对第一人脸图像进行关键点检测，得到第一人脸图像中的关键点，进而依据关键点确定第一人脸图像中苹果肌所在的目标区域，并基于关键点确定调整第一人脸图像的苹果肌的调整参数，利用该调整参数调整该目标区域，就能够生成苹果肌调整后的第二人脸图像。如此，本公开实施例提供的技术方案，至少提供了对人脸图像中的苹果肌进行调整的功能，后续可以根据用户需求调整人脸图像中的苹果肌，从而提升对人脸图像美化和滑稽化的效果。
附图说明
为了更清楚地说明本申请实施例或背景技术中的技术方案，下面将对本申请实施例或背景技术中所需要使用的附图进行说明。
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1为本公开实施例提供的一种像素坐标系的示意图；
图2为本公开实施例提供的一种人脸图像的区域划分的示意图；
图3为本公开实施例提供的一种图像处理方法的流程示意图；
图4为本公开实施例提供的一种人脸关键点的示意图；
图5为本公开实施例提供的另一种图像处理方法的流程示意图；
图6A为本公开实施例提供的一种正脸人像的效果示意图；
图6B为本公开实施例提供的一种侧脸人像的效果示意图；
图6C为本公开实施例提供的另一种侧脸人像的效果示意图；
图7为本公开实施例提供的一种掩码图像的效果示意图；
图8为本公开实施例提供的一种图像处理装置的结构示意图；
图9为本公开实施例提供一种图像设备的结构示意图；
图10为本公开实施例提供的一种图像处理方法的效果示意图。
具体实施方式
以下结合说明书附图及具体实施例对本公开的技术方案做进一步的详细阐述。
在进行接下来的阐述之前，首先对本公开实施例中的像素坐标系进行定义。如图1所示，以人脸图像A的右下角为像素坐标系的原点o、平行于人脸图像A的行的方向为x轴的方向、平行于人脸图像A的列的方向为y轴的方向，构建像素坐标系xoy。在像素坐标系下，横坐标用于表示人脸图像A中的像素在人脸图像A中的列数，纵坐标用于表示人脸图像A中的像素在人脸图像A中的行数，横坐标和纵坐标的单位均可以是像素。例如，假设图1中的像素a的坐标为(10，30)，即像素a的横坐标为10个像素，像素a的纵坐标为30个像素，像素a为人脸图像A中的第10列第30行的像素。
为表述方便，本公开实施例将人脸图像中的人脸区域划分为左脸区域和右脸区域。如图2所示，以人脸区域的对称中心线为基准将人脸区域分为左脸区域和右脸区域。
如图3所示，本实施例提供一种图像处理方法，包括：
步骤S110：检测第一人脸图像的关键点；
步骤S120：基于所述关键点，确定苹果肌调整的目标区域；
步骤S130：基于所述关键点，确定苹果肌的调整参数；
步骤S140：基于所述调整参数，调整所述目标区域形成第二人脸图像。
本实施例中，人脸中的苹果肌可以理解为在眼睛下方区域中呈倒三角状的组织所在区域。在人微笑或做表情时会因为脸部肌肉的挤压而稍稍隆起，该呈倒三角状的组织所在区域看起来就像圆润有光泽的苹果，故称为苹果肌。本实施例中提供了一种对苹果肌的调整方案，实际应用中还可适用在人脸其它部位或者卡通人物的脸部部位或者动物的脸部部位等，基于该调整方案的技术构思对于以上脸部部位的调整，均可视为本公开实施例所保护的技术范围内。
在步骤S110中利用神经网络等深度学习模型或者人脸关键点检测算法对第一人脸图像进行处理，确定第一人脸图像中的人脸关键点。上述人脸关键点检测算法可以是OpenFace、多任务级联卷积神经网络(multi-task cascaded convolutional networks，MTCNN)、调整卷积神经网络(tweaked convolutional neural networks，TCNN)、或任务约束深度卷积神经网络(tasks-constrained deep convolutional network，TCDCN)中的一种，本公开对人脸关键点检测算法不做限定。
例如，所述关键点包括但不限于以下关键点：
脸型轮廓关键点，例如，参见图4所示的关键点0到关键点32；
眉毛关键点，例如，参见图4的关键点33至37、关键点38至42以及关键点64至关键点71；
眼睛关键点；
鼻子关键点；
唇部关键点等，所述唇部关键点可参见图4的关键点84至关键点103。
所述眼睛关键点根据关键点在眼睛上的分布，又可以包括：
位于内眼角的眼头关键点；例如，参考图4所示的关键点55和关键点58；
位于外眼角的眼尾关键点；例如，参考图4中所示的关键点52和关键点62；
位于上眼睑的上眼睑关键点；例如，参考图4所示的关键点53、72及54，及关键点59、75及60；
位于下眼睑的下眼睑关键点；例如，参见图4所示的关键点62、76及63；关键点57、73及56；
位于眼珠的眼珠关键点，例如，参见图4所示的关键点74、104及关键点77及105；
所述鼻子关键点根据关键点在鼻子上的分布，可以分为：
位于山根的关键点，例如，参考图4所示的关键点43；
位于鼻梁的鼻梁关键点；例如，参考图4的关键点44和关键点45；
位于鼻翼的鼻翼关键点；例如，参考图4的关键点80至83；
位于鼻头的鼻头关键点；例如，参考图4的关键点46；
位于鼻底的鼻底关键点；例如，参考图4所示的关键点47至51。
在本实施例中，与苹果肌调整相关的关键点包括以下至少之一：
如图4所示的脸型轮廓关键点中靠近颧骨位置的关键点4、5、28及27；
眼尾关键点；
眼头关键点；
在步骤S130中首先根据关键点中的一个或多个，在第一人脸图像中圈定出进行苹果肌调整的目标区域。在圈定该区域之后可以依据该区域内的像素实现苹果肌的调整。具体实施中可从以上脸型轮廓关键点中选取第一脸型轮廓关键点和/或第二脸型轮廓关键点，以便于确定苹果肌的调整参数。
在本实施例中，所述目标区域可为圆形区域或者椭圆形区域。
在步骤S130中会根据关键点中的一个或多个关键点，确定苹果肌的调整参数。所述苹果肌的调整参数包括但不限于以下至少之一：
苹果肌的调整方向；
苹果肌的调整幅度；
苹果肌的调整梯度。
调整方向，可以用于确定苹果肌在第一人脸图像上的移动方向；
调整幅度，可以用于确定苹果肌在第一人脸图像上的改变尺度，例如，移动距离或缩放比例等；
调整梯度，可以用于在多次调整时不同次调整幅度差异，或者，进行苹果肌的分区域调整时，不同区域之间的调整幅度差。例如，将圆形或椭圆形的目标区域划分为内区域和外区域；外区域包围在内区域内，在对苹果肌的亮度调整时，内区域的亮度调整幅度大于外区域的亮度调整幅度，则内外区域的调整亮度之差就构成了所述调整梯度。当然此处仅是举例，具体的实现方式有多种。
在步骤S130中用于确定所述调整参数和所述目标区域的关键点可以相同也可以不相同。在一些实施例中，用于确定所述调整参数和所述目标区域的关键点可以部分相同。例如，可以使用相同的脸部轮廓关键点来确定所述调整参数和所述目标区域。再例如，可以同时使用相同的眼部关键点来确定所述调整参数和所述目标区域。
在步骤S140中会根据调整参数，对目标区域在第一人脸图像中的位置、形状、尺寸、或颜色中的至少一个进行调整，从而形成第二人脸图像。
具体地，所述步骤S140可包括以下至少之一：
对所述目标区域进行移动，调整所述目标区域的位置；
对所述目标区域进行尺寸缩放，调整所述目标区域在第二人脸图像上的尺寸；
对所述目标区域进行颜色变换，调整所述目标区域的颜色，从而形成人脸高光或阴影区域等；
对所述目标区域内进行形状变化，调整所述目标区域呈现的形状，从而使得成像与人脸的脸型相适配的苹果肌形状。例如，不同人像成像不同的脸型，例如，有的是鹅蛋型脸、有的是心形脸、有的圆脸、有的是棱形脸。苹果肌的大致形状都是倒三角形；但是倒三角形的三个角的角度不同或顶点在人脸图像上的位置不同，会使得苹果肌呈现不同的视觉感受。故在一些实施例中，图像设备还可以通过调整目标区域内的像素的位置等方式，使得苹果肌在目标区域内或移动后的目标区域内呈现出不同形状的倒三角形，以与脸型相适配，从而实现人像美化、或滑稽化等处理。
总之，在本实施例中提供了一种方法可以对人脸图像中的苹果肌进行调整，从而实现人脸图像中苹果肌的美化、滑稽化、可爱化或者不同风格类型的苹果肌；如此，可以根据用户需求实现图像的苹果肌处理，从而满足用户不同图像求，提升用户体验，和图像处理质量。
在一些实施例中，所述步骤S130生成所述调整参数时，可以根据当前苹果肌的预期调整效果，生成所述调整参数。例如，当前预期调整效果为苹果肌美化效果，则根据美化效果使得苹果肌在第二人脸图像相对于第一人脸图像上呈现出大众审美的美化效果。再例如，当前预期调整效果为滑稽化效果，以通过苹果肌在第一人脸图像上的调整，呈现出幽默的效果；则在生成所述调整参数时，将结合所述关键点和所述滑稽化效果生成使得第二人脸图像呈现幽默搞笑效果的苹果肌。总之，在一些实施例中，所述调整参数的生成还会根据所述预期调整效果来确定。
在本实施例中，对苹果肌的调整可以用于图像美化过程中，在图像美化的应用程序中，本公开实施例提供的苹果肌调整可以用于一键式人像美图中，如此，苹果肌美化、会和眼睛美化、鼻子美化或脸型美化等一同作为一键式人像美图的一个功能被实现。
当然本实施例提供的苹果肌的调整，也可以用于专门的苹果肌美化功能中。例如，在美图应用中设置有苹果肌美化控件，若检测到用户作用于该苹果肌美化控件的操作时，单独对图像中的苹果肌使用上述方法进行美化。
总之，在本实施例中所述的图像处理方法，具有苹果肌的调整功能，在进行人像处理的过程中可以进行苹果肌调整，从而满足用户苹果肌调整的需求，提升了用户体验和图像设备的智能性。
在另一些实施例中，如图5所示，所述方法包括：
步骤S111：确定所述第一人脸图像中人脸的朝向；
所述步骤S120可包括步骤S121；所述步骤S121可包括：基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
在本实施例中，还会确定第一人脸图像的朝向。第一人脸图像的朝向至少分为两种：
第一种：正脸朝向，正脸朝向表示第一人脸图像为正脸图像；
第二种：侧脸朝向，侧脸朝向表示第一人脸图像为侧脸图像。
在采集上述第一人脸图像的成像设备的拍摄方向与过被拍摄人物的脸部区域的对称中心线之间的夹角(下文将称为人脸偏转角)处于第一预设范围内时，第一人脸图像的朝向为正脸朝向。
上述人脸偏转角指成像设备的拍摄方向与被拍摄的人物的脸部区域的竖直线之间的夹角，且从被拍摄人物的头顶从上往下看，成像设备的拍摄方向相较于被拍摄人物的脸部区域的对称中心线的偏移方向为顺时针方向时，人脸偏转角为正，反之，从被拍摄人物的头顶从上往下看，成像设备的拍摄方向相较于被拍摄人物的脸部区域的对称中心线的偏移方向为逆时针方向时，人脸偏转角为负。
在上述第一人脸图像的人脸偏转角处于第二预设范围内时，第一人脸图像的朝向为侧脸朝向。
上述第一预设范围与上述第二预设范围互为补集。例如，第一预设范围为-10度至10度，则第二预设范围为-180度至-10度，以及10度至180度。
人脸图像中所述正脸朝向以外的朝向都可以视为侧脸朝向，当然此时不包含有人脸的背影图像除外。
图6A为一种正脸朝向的人脸图像；图6B为一种侧脸朝向的人脸图像；图6C为另一种侧脸朝向的人脸图像。
在一些实施例中，图6A所示的人脸图像的人脸偏转角为0度，假设图6B所示的人脸图像的人脸偏转角为-90度，图6C所示的人脸图像的人脸偏转角为90度。
当然上述第一预设范围和上述第二预设范围可以根据需要进行设置。
由于正脸朝向的人脸图像(下文将称为正脸图像)中的苹果肌所覆盖的区域和侧脸朝向的人脸图像(下文将称为侧脸图像)中的苹果肌所覆盖的区域不同(包括苹果肌覆盖的区域的面积不同、苹果肌覆盖的区域的角度不同等等)，若对正脸图像和侧脸图像采用相同的调整标准或调整参数，将导致对人脸图像中的苹果肌的调整效果不佳(如：调整后的苹果肌区域的面积与人脸区域的面积的比值不合适，又如：调整后的苹果肌区域的颜色和人脸区域中非苹果肌区域的颜色的差别较大)。为此，本公开实施例对不同朝向的人脸图像采用不同的方式确定调整参数，以提升对人脸图像的苹果肌的调整效果。
以下分别介绍正脸图像和侧脸图像在确定目标区域时的异同：
针对于正脸图像而言，所述步骤S120可包括：
在确定所述朝向表示所述第一人脸图像为正脸图像的情况下，基于第一脸型轮廓关键点和鼻翼关键点，确定所述目标区域的第一中间点。
所述第一脸型轮廓关键点可为纵坐标处于第一目标范围内的脸型轮廓关键点，其中，第一目标范围为苹果肌覆盖的区域的纵坐标范围与第三预设范围的和。可选的，上述第三预设范围为-5个像素至5个像素。举例来说，若苹果肌覆盖的区域内的像素的纵坐标的最大值为35个像素，最小值为16个像素，第三预设范围为-4个像素至5个像素，则纵坐标大于或等于16-4＝12个像素且小于或等于35+5＝40个像素的人脸轮廓关键点为第一人脸轮廓关键点。
可选的，参考图4所示，关键点4及关键点5可作为调整人脸图像中左苹果肌(即位于左脸区域的苹果肌)的第一脸型轮廓关键点。
在本实施例中，优选关键点5作为所述第一脸型轮廓关键点。所述鼻翼关键点为纵坐标与第一脸型轮廓关键点的纵坐标的差值在第四预设范围内，且位于鼻翼区域(该区域可根据用户需求进行定义)内的关键点。例如，假设第一脸型轮廓关键点中的关键点的纵坐标的最大值为50个像素、最小值为40个像素，第四预设范围为10个像素，则鼻翼关键点为纵坐标在大于或等于30个像素且小于或等于60个像素的范围内的鼻翼关键点。
可选的，上述鼻翼关键点可以是图4所示的关键点80或关键点82，在本实施例中优选选用关键点80作为所述鼻翼关键点。可选的，若第一人脸图像为正脸图像，则选用图4中所示的关键点和关键点80分别作为调整右苹果肌(即位于右脸区域的苹果肌)的第一脸型轮廓关键点和鼻翼关键点，调整效果更加满足用户期望的。选用图4中所示的关键点27和关键点81分别作为左脸苹果肌的第一脸型轮廓关键点和鼻翼关键点。
所述第一中间点包括但不限于：所述第一脸型轮廓关键点和鼻翼关键点的中点(如图4所示的关键点80和关键点82之间的中点、关键点81和关键点83之间的中点)。
在一些实施例中，第一人脸图像呈现的人脸图像的人脸特征不同，则基于第一脸型轮廓关键点和鼻翼关键点的中点进行位置修正之后，得到所述第一中间点。例如，若第一人脸图像呈现的人脸A，并假设眼珠关键点到对应位置的第一脸型轮廓关键点之间的距离为第一距离，且假设眼珠关键点到对应位置处的鼻翼关键点的中点的距离为第二距离。上述对应位置处的第一脸型轮廓关键点指：若眼珠关键点位于人脸区域的左脸区域，与眼珠关键点对应的第一脸型轮廓关键点为位于左脸区域的第一人脸关键点中距离眼珠关键点最近的第一脸型轮廓关键点；若眼珠关键点位于人脸区域的右脸区域，与眼珠关键点对应的第一脸型轮廓关键点为位于右脸区域的第一人脸轮廓关键点中距离眼珠关键点最近的第一脸型轮廓关键点。上述对应位置的鼻翼关键点的中点指：若眼珠关键点位于人脸区域的左脸区域，与眼珠关键点对应的鼻翼关键点的中点为位于左脸区域的鼻翼关键点的中点；若眼珠关键点位于人脸区域的右脸区域，与眼珠关键点对应的鼻翼关键点的中点为位于右脸区域的鼻翼关键点的中点。
并可根据该比值生成所述修正参数。若第一距离和第二距离的比值为1，则将与眼珠关键点对应的第一脸型轮廓关键点和与眼珠关键点对应的鼻翼关键点的中点作为所述第一中间点。
若第一距离和第二距离之间的比值大于1或小于1，可先确定与眼珠关键点对应的第一脸型轮廓关键点和与眼珠关键点对应的鼻翼关键点的连线上的中点(下文将称为第一待确认中间点)向对称中心线移动或者向脸部轮廓线的偏移量，该偏移量包括横向偏移量和纵向偏移量。将由第一待确认中间点的坐标与偏移量的和确定的点作为第一中间点。
在一种可能实现的方式中，若第一距离不等于第二距离，可通过偏移量使待确认中间点向脸部轮廓线移动，如：若眼珠关键点和第一脸型轮廓关键点均位于左脸区域，则可将横向偏移量取为负值，以使第一待确认中间点向脸部轮廓线移动；若眼珠关键点和第一脸型轮廓关键点均位于左脸区域，则可将横向偏移量取为正值，以使第一待确认中间点向对称中心线移动；若眼珠关键点和第一脸型轮廓关键点均位于右脸区域，则可将横向偏移量取为正值，以使第一待确认中间点向脸部轮廓线移动；若眼珠关键点和第一脸型轮廓关键点均位于右脸区域，则可将横向偏移量取为负值，以使第一待确认中间点向对称中心线移动。
若目标区域为圆形区域，依据用户设置的目标区域的半径和第一中间点即可在人脸区域中确定目标区域。在一种确定目标区域的半径的实现方式中，目标区域的半径可是根据人脸区域的面积确定。在另一种确定目标区域的半径的实现方式中，目标区域的半径还可以由深度学习模型根据第一人脸图像估算获得。
进一步地，所述步骤S120还包括：
基于所述第一中间点及眼尾关键点，确定所述目标区域。
在一种可能实现的方式中，利用线性插值算法以第一中间点和眼尾关键点在第一人脸图像中的坐标为已知量，计算得到目标区域的圆形半径或者长轴和短轴。若目标区域为圆形，以第一中间点为圆形区域的圆心、计算得到的目标区域的圆形半径为半径，可构建目标区域。若目标区域为椭圆形，第一中间点为椭圆形区域的中心，长轴或短轴之一确定了，再基于预先设定的长轴和短轴之间的比值，另一个轴也就确定了，若中心、长轴和短轴都确定了，自然椭圆形的目标区域也就确定了。
以上仅是仅是线性插值算法的举例，具体实现时不局限于上述任意一种。
针对侧脸图像，所述步骤S120可包括：
在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于第二脸型轮廓关键点和鼻翼关键点，确定所述目标区域的第二中间点。
在进行苹果肌调整时，由于人脸朝向的不同，可以选择不同的关键点来确定目标区域。
例如，侧脸图像中的第二脸型轮廓关键点是不同于正脸图像的第一脸型轮廓关键点。第二脸型轮廓关键点可为纵坐标处于第二目标范围内的脸型轮廓关键点，其中，第二目标范围为苹果肌覆盖的区域的纵坐标范围与第五预设范围的和，第五预设范围小于上述第三预设范围。可选的，图4所示的关键点4、5、27及28均为第二脸型轮廓关键点。
在本实施例中，所述鼻翼关键点可与正脸图像的鼻翼关键点相同，可以选择参考图4所示的关键点80或关键点81。
在本实施例中，所述第二中间点同样可为第二脸型轮廓关键点和鼻翼关键点的中点(下文将称为第二待确认中间点)，或者参考正脸图像中利用偏移量移动该第一待确认中间点得到第一中间点的方式移动侧脸图像中的第二待确认中间点得到第二中间点。
第二中间点可为目标区域的中心点。
进一步地，所述步骤S120还可包括：基于所述第二中间点与眼尾关键点，确定所述目标区域的范围。
在本实施例中，所述第二中间点和眼尾关键点同样可以基于线性插值算法得到所述目标区域的半径等确定范围的参数，从而在所述第一人脸图像上确定出所述目标区域的范围。
在一些实施例中，所述步骤S130可包括：
基于所述关键点及所述朝向，确定所述苹果肌的调整参数。
为了实现苹果肌的优化调整，还可基于关键点和朝向来确定具体的调整参数。
不同的朝向，苹果肌的调整策略可能不同。例如，正脸图像的调整和侧脸图像的调整，调整方向和/或角度是有不同，对应的调整策略也不同。再例如，所述朝向不同，所述苹果肌的亮度调整不同和/或调整梯度不同等。
具体地如，所述步骤S130具体可包括：基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
所述朝向不同，苹果肌的调整方向不同。
在进行图像美化时，一般需要进行苹果肌的上拉或者隆起调整，由于人脸朝向不同，则上拉的具体方向或者隆起的具体方向不同。
在一些实施例中，所述基于所述关键点及所述朝向，确定所述苹果肌的调整方向，包括：
在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第一方向；
在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
在一些实施例中，所述第三方向不同于所述第二方向，包括但不限于：第三方向垂直于所述第二方向。
例如，针对于侧脸图像而言会对目标区域进行两个方向上的移动，一个是使得目标区域向眼睛方向上拉，另一个是使得目标区域向脸边缘外移，可选的，第三方向垂直第二方向。
但是为了优化苹果肌的调整效果，在本实施例中，所述第一方向可为所述第一中间点指向所述眼尾关键点的方向；所述第二方向可为所述第二中间点指向所述眼尾关键点的方向；所述第三方向可为：第二脸部轮廓关键点指向鼻翼关键点的反方向(即，从所述鼻翼关键点指向所述第二脸部轮廓关键点的方向)。
由于脸型轮廓关键点和眼尾关键点都是从第一人脸图像中检测出来的，其分布本身就反应了人脸的特点，如此，基于脸型轮廓关键点和眼尾关键点确定调整的第一方向、第二方向和第三方向，实质上可以针对不同的人脸进行个性调整，实现苹果肌的优化调整。
具体地，所述步骤S130可包括：
基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离。
所述眼尾关键点和眼头关键点之间的距离实质上就是第一人脸图像中眼睛的长度；例如，眼尾关键点和眼头关键点之间隔离M个像素；则所述最大距离可为M*A个像素。A可为小于1的正数。进一步地，例如，所述A的取值可为0.44至0.74之间的正数。
在一些实施例中，所述苹果肌的实际移动距离可以基于用户操作确定，例如，用户在人机交互界面输入用户操作，根据用户操作确定出实际移动距离。若基于用户操作计算得到的移动距离大于所述最大移动距离，则将所述最大移动距离作为所述苹果肌的实际移动距离对苹果肌进行调整。
在一些实施例中，所述步骤S130可包括：
若所述朝向表示第一人脸图像为侧脸图像，根据人脸偏转角及所述最大移动距离，确定侧脸图像的苹果肌的实际移动距离。
在本实施例中，基于关键点可确定苹果肌的最大移动距离，该最大移动距离也即目标区域的最大移动距离。
在本实施例中，以眼睛的关键点为所述关键点，基于眼睛的尺寸确定出苹果肌的最大移动距离。
在一种可能实现的方式中，上述实际移动距离与侧脸图像中的人脸偏转角呈正相关。例如，在一些实施例中，当人脸偏转角为90度或-90度时，实际移动距离可为上述最大移动距离。当人脸偏转角不等于90度和-90度时，实际移动距离＝上述最大移动距离*a，其中，a为小于1的正数。
当然以上仅是举例说明，具体的实现方式有很多种，不局限于以上任意一种。
在一些实施例中，所述方法还包括：
步骤S150：对第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
第二人脸图像相对于第一人脸图像使得苹果肌在人脸上进行了移动。在本实施例中，还可对移动后的苹果肌进行提亮处理，使得苹果肌看起来更加充盈和饱满。
在一些实施例中，在上述提亮处理中，苹果肌中不同位置的像素的提亮程度不同。例如，以苹果肌的最高点位置向苹果肌的周边提亮的程度依次降低，从而使得苹果肌的最高点的位置亮度最大，然后依次缓慢递减到与苹果肌周边皮肤像素相同的亮度，一方面使得苹果肌调整之后看起来更加自然，同时利用苹果肌不同位置的亮度差体现出苹果肌的立体效果。
在一些实施例中，所述方法还包括：
从所述第二人脸图像中获取人脸的肤色参数；
基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
所述步骤S150可包括：基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，以获得所述第三人脸图像。
一种方式，为了减少针对不同的人脸图像都采用单一的提亮参数进行提亮，导致原本较暗的人脸图像，提亮后出现苹果肌非常不自然的现象，而针对原本肤色较亮的人脸图像，提亮后苹果肌的提亮效果不明显的现象。在本实施例中，会获取人脸图像的肤色参数，该肤色参数包括但不限限于以下至少之一：
人脸图像中皮肤像素的颜色值和人脸图像中眼睛和唇部之间的皮肤像素的颜色值、人脸图像中皮肤像素的灰度直方图和人脸图像中眼睛和唇部之间的皮肤像素的颜色值、人脸图像中皮肤像素的颜色值和人脸图像中眼睛和唇部之间的皮肤像素的灰度直方图、人脸图像中皮肤像素的灰度直方图和人脸图像中眼睛和唇部之间的皮肤像素的灰度直方图。
基于该肤色参数可确定苹果肌和苹果肌周围的皮肤的亮度平均值或亮度中值等，进而可基于该平均值或亮度中值确定苹果肌区域的像素的提亮参数，以区分苹果肌和苹果肌周边的皮肤，同时也可减小苹果肌和苹果肌周边的皮肤差异。
该提亮参数包括但不限于以下至少之一：提亮幅度，提亮比例。
提亮苹果肌的具体实现方式有多种，在本实施例中，会利用掩码图像进行提亮。
在一些实施例中，所述苹果肌提亮可包括：
基于该提亮幅度可以在苹果肌原本的像素值加上一个对应的提亮幅度，就得到提亮后的像素值，在完成对苹果肌区域内所有像素的提亮处理后，可得到提亮后的苹果肌，并获得第三人脸图像；
在另一些实施例中，所述苹果肌提亮可包括：
基于该提亮比例可以在苹果肌原本的像素值上乘上该提亮比例就得到了提亮后的像素值，在完成对苹果肌区域内所有像素的提亮处理后，可得到提亮后的苹果肌，并获得第三人脸图像。
在一些实施例中，所述步骤S150可包括：根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
所述步骤S150可包括：
确定苹果肌在第二人脸图像中的所在位置；
基于该所在位置形成一个与所述第二人脸图像同尺寸的掩码图像；该掩码图像的在第二人脸图像所在的位置形成有半透明的提亮光晕；在其他位置为透明区域；将所述掩码图像叠加在所述第二人脸图像上生成所述第三人脸图像。
此时，所述掩码图像中的提亮光晕的亮度值直接可为苹果肌提亮后的亮度值，故所述提亮参数还可包括：苹果肌提亮后的亮度值。
图7所示为本实施例提供一张掩码图像的示意图。在图5中白色光晕区域就是用于与调整后的苹果肌所在区域重叠的区域，以提亮苹果肌。
进行苹果肌提亮的另一种方式是：所述根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像，包括：
根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的掩码图像；
和/或，
所述基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像，包括：
将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
基于所述第二像素值，生成所述第三人脸图像。
在一些实施例中，苹果肌的亮斑可为预先确定的，该亮斑的形状和亮度都可以预先确定。如此，可以加速掩码图像的生成速率。
苹果肌的亮斑在掩码图像中的位置，是与第二人脸图像中苹果肌所在的位置相对应的。例如，在第一人脸图像中进行关键点的检测，由于第一人脸图像和第二人脸图像仅是苹果肌发生了一些变换，但是变换还是在人脸范围内，且不会是巨大的移动，故可以依据步骤S110中检测到关键点来确定出第二人脸图像中苹果肌所在的区域；例如，以步骤S120中确定的目标区域；或者，基于目标区域和实际移动距离，得到第二人脸图像中的苹果肌所在的区域，基于该区域将所述亮斑对应的添加到掩码图像中。
在本实施例中，所述掩码图像中除了亮斑所在位置的像素值会大于所述预定阈值，亮斑所在位置以外像素的像素值都会小于或等于所述预定阈值。如此在进行像素混合时，可以利用图像处理器(GPU)等适用于大量运算的运算模块，在遍历掩码图像中的像素时，通过像素值与预定阈值的比较，确定该像素时否需要与第二人脸图像中的像素进行混合。例如，第二人脸图像为RGB图像，则第二人脸图像包括3个颜色通道，这3个颜色通道中每一个像素都有其对应的像素值，若掩码图像中第M个像素的像素值大于所述预定阈值，则将掩码图像中第M个像素的像素值和所述第二人脸图像中第M个像素三个颜色通道的颜色值进行混合，再组合这3个颜色通道混合后的值就得到了所述第二像素值。此处的混合，可以为线性混合，也可以是非线性混合。在本实施例中，所述掩码图像和第二人脸图像的混合优选为非线性混合，如此得到的苹果肌的提亮效果更佳。
具体的所述非线性混合的函数关系可如下：
Sqrt(original color)*(a*mask color-1.0)+b*original color*(c-mask color)，其中，original color为第二人脸图像中的第一像素值；mask color为掩码图像中的像素值；a，b，c均为已知的计算参数。具体取值可以根据需求进行设定。
Sqrt表示开平方，Sqrt(original color)表示对第一像素值开平方。
在本实施例中，所述a的取值和b的取值可以相同，例如，在original color和mask color都被归一化0到1之间的数值之后，所述a和b的取值均可为2.0；所述c的取值可为1.0。
总之，所述a、b及c的取值可以相同，也可以不相同。
在一些实施例中，在进行像素值混合之前会对像素值或颜色值进行归一化处理，得到的像素值或颜色值均是位于0到1之间的数据。此时，一方面，即便不同比特数目的颜色通道形成的图像，也会得到同样位于0到1之间的像素值或颜色值。另一方面，通过归一化处理，则大大缩小了后续参与像素混合计算的数值，从而简化了计算。
在一些实施例中，所述预定阈值可为0.4、0.45、0.5、0.55及0.6等取值。所述预定阈值的取值范围可为0.4至0.6之间。
进一步地，所述方法还包括：获取控制参数；
所述基于所述第二像素值，生成所述第三人脸图像，包括：
基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素值；
基于所述第三像素值，生成所述第三人脸图像。
在本实施例中所述控制参数可为图像设备从人机交互接口或者从其他设备接收的外部控制参数，可以用于控制苹果肌的提亮程度，是一种苹果肌提亮的程度控制参数。
在得到第二像素值之后，再基于控制参数与第一像素值进行线性混合，将得到所述第三像素值。例如，所述控制参数为比例参数，可以基于A(RGB1)+RGB2＝RGB3。例如，A可为所述控制参数；所述RGB1可为第二人脸图像原始的第一像素值；RGB2可为混合后的所述第二像素值；RGB3可为再次混合后形成所述第三人脸图像的第三像素值。
在本实施例中不是直接利用第二像素值生成所述第三人脸图像，而是会利用原始的第二人脸图像和混合后的第二像素值进行再次线性混合，得到所述第三人脸图像，如此，可以使得苹果肌的提亮更加自然，图像处理效果更加优越。
需要理解的是，本公开实施例详细阐述了如何调整人脸图像中的苹果肌的目标区域，而在实际应用中还可依据本公开实施例提供的技术方案对人脸图像中的额头区域、下颚区域、颧骨区域等进行调整。
如图8所示，本实施例提供一种图像处理装置1，包括：
检测单元11，用于检测第一人脸图像的关键点；
第一确定单元12，用于基于所述关键点，确定苹果肌调整的目标区域；
第二确定单元13，用于基于所述关键点，确定苹果肌的调整参数；
调整单元14，用于基于所述调整参数，调整所述目标区域形成第二人脸图像。
结合本公开任一实施方式，所述装置1还包括：
第三确定单元15，用于确定所述第一人脸图像中人脸的朝向；
所述第一确定单元12用于：
基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
结合本公开任一实施方式，所述关键点包括：眼尾关键点、第一脸型轮廓关键点和鼻翼关键点；
所述第二确定单元13用于：
在确定所述朝向表示第一人脸图像为正脸图像的情况下，基于所述第一脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第一中间点；
基于所述第一中间点及所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述关键点包括眼尾关键点、第二脸型轮廓关键点和鼻翼关键点；
所述第一确定单元12用于：
在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于所述第二脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第二中间点；
基于所述第二中间点与所述眼尾关键点，确定所述目标区域。
结合本公开任一实施方式，所述第二确定单元13用于：
基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
结合本公开任一实施方式，所述第三确定单元15用于：
在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第一方向。
结合本公开任一实施方式，所述第三确定单元15还用于：
在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
结合本公开任一实施方式，所述第一方向为：所述目标区域的第一中间点指向眼尾关键点的方向。
结合本公开任一实施方式，所述第二方向为：所述目标区域的第二中间点指向眼尾关键点的方向；
所述第三方向为：鼻翼关键点指向确定目标区域的第二脸型轮廓关键点的方向。
结合本公开任一实施方式，所述第二确定单元13用于：
基于所述关键点和所述朝向，确定所述苹果肌的调整幅度。
结合本公开任一实施方式，所述第二确定单元13用于：
基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离；
在所述朝向表示所述第一人脸图像为侧脸图像的情况下，根据人脸偏转角及所述最大移动距离，确定所述第一人脸图像的苹果肌的实际移动距离。
结合本公开任一实施方式，所述装置1还包括：
提亮处理单元16，用于对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
结合本公开任一实施方式，所述提亮处理单元16用于：
从所述第二人脸图像中获取人脸的肤色参数；
基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，获得所述第三人脸图像。
结合本公开任一实施方式，所述提亮处理单元16用于：
根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；
基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
结合本公开任一实施方式，所述提亮处理单元16用于：
根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的所述掩码图像。
结合本公开任一实施方式，所述提亮处理单元16用于：
将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
基于所述第二像素值，生成所述第三人脸图像。
结合本公开任一实施方式，所述装置1还包括：
获取单元17，用于获取控制参数；
所述提亮处理单元16用于：
基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素值；
基于所述第三像素值，生成所述第三人脸图像。
以下基于上述实施例提供两个具体示例：
示例1：
苹果肌调整方法，可包括以下步骤：
1)将第一人脸图像中的的苹果肌范围区域向上提拉；
2)在第一人脸图像为侧脸图像的时候除了向上提拉，同时将这一块区域根据第一人脸图像的人脸偏转角向外膨胀；
3)使用掩码(mask)纹理将苹果肌所在区域提亮。
如图9所示，对第一人脸图像的苹果肌所在区域进行苹果肌上拉或横向膨胀的操作，得到第二人脸图像。在图9中第二人脸图像相对于第一人脸图像的虚线圆形区域可为所述苹果肌所在的目标区域；将第二掩码图像和第二人脸图像合并之后得到苹果肌被提亮的第三人脸图像。
示例2：
以第一人脸图像中的左脸区域为例，提供一种苹果肌调整的示例，包括：
一、在第一人脸图像为正脸图像的情况下，将第一人脸图像的苹果肌范围的区域(对应于上述目标区域)向上提拉；
1)计算关键点5与关键点80的中点作为拉动区域(圆形区域)的中间点，拉动区域的最大移动距离设置为关键点52和关键点55距离的k倍，k为正整数，可基于实际需求进行配置；
2)向上提拉的方向为拉动区域的中间点指向关键点52(目标点)的方向，拉动区域的范围(半径)由中间点到目标点线性插值得到；
二、在第一人脸图像为侧脸图像的情况下，除了对苹果肌所在区域向上提拉外，同时将这一块区域根据第一人脸图像的人脸偏转角向外膨胀；
1)依据第一人脸图像的人脸偏转角可确定一个膨胀区域，膨胀区域以关键点4为中间点，膨胀区域的最大移动距离与提拉距离相同，同时与人脸偏转角正相关。
2)膨胀区域以关键点4为中间点，膨胀方向为关键点4指向关键点80的反方向，膨胀区域的范围(半径)由中间点到目标点线性插值得到；
三、使用mask纹理将苹果肌区域提亮。
1)最后进行提亮处理，使用一张mask纹理，该纹理将标准人脸苹果肌区域标亮，再与第一人脸图像进行颜色的混合，这样人脸苹果肌区域会被提亮，看起来更加立体。
如图10所示，本实施例提供了一种图像设备，包括：
存储器；
处理器，与所述存储器连接，用于通过执行位于所述存储器上的计算机可执行指令，能够实现前述任意技术方案提供的图像处理方法，例如，图1和/或图3所示图像处理方法。
该存储器可为各种类型的存储器，可为随机存储器、只读存储器、闪存等。所述存储器可用于信息存储，例如，存储计算机可执行指令等。所述计算机可执行指令可为各种程序指令，例如，目标程序指令和/或源程序指令等。
所述处理器可为各种类型的处理器，例如，中央处理器、微处理器、数字信号处理器、可编程阵列、数字信号处理器、专用集成电路或图像处理器等。
所述处理器可以通过总线与所述存储器连接。所述总线可为集成电路总线等。
在一些实施例中，所述图像设备还可包括：通信接口，该通信接口可包括：网络接口、例如，局域网接口、收发天线等。所述通信接口同样与所述处理器连接，能够用于信息收发。
在一些实施例中，所述图像设备还包括人机交互接口，例如，所述人机交互接口可包括各种输入输出设备，例如，键盘、触摸屏等。
本实施例提供一种计算机存储介质，所述计算机存储介质存储有计算机可执行指令；所述计算机可执行指令被执行后，能够应用于图像设备、数据库、第一私有网络中一个或多个技术方案提供的图像处理方法，例如，图1和/或图2所示图像处理方法。
所述计算机存储介质可为包括具有记录功能的各种记录介质，例如，CD、软盘、硬盘、磁带、光盘、U盘或移动硬盘等各种存储介质。可选的所述计算机存储介质可为非瞬间存储介质，该计算机存储介质可被处理器读取，从而使得存储在计算机存储机制上的计算机可执行指令被处第一理器获取并执行后，能够实现前述任意一个技术方案提供的图像处理方法，例如，执行应用于图像设备中的图像处理方法或应用服务器中的图像处理方法。
本实施例还提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令；所述计算机可执行指令被执行后，能够实现前述一个或多个技术方案提供的图像处理方法，例如，例如，图3和/或图5所示的图像处理方法。
所述包括有形地包含在计算机存储介质上的计算机程序，计算机程序包含用于执行流程图所示的方法的程序代码，程序代码可包括对应执行本发明实施例提供的方法步骤对应的指令。
在本公开所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元，即可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本公开各实施例中的各功能单元可以全部集成在一个处理模块中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (36)

  1. 一种图像处理方法，其特征在于，包括：
    检测第一人脸图像的关键点；
    基于所述关键点，确定苹果肌调整的目标区域；
    基于所述关键点，确定苹果肌的调整参数；
    基于所述调整参数，调整所述目标区域形成第二人脸图像。
  2. 根据权利要求1所述的方法，其特征在于，在所述基于所述关键点，确定苹果肌调整的目标区域之前，所述方法还包括：
    确定所述第一人脸图像中人脸的朝向；
    所述基于所述关键点，确定苹果肌调整的目标区域，包括：
    基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
  3. 根据权利要求2所述的方法，其特征在于，所述关键点包括：眼尾关键点、第一脸型轮廓关键点和鼻翼关键点；
    所述基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域，包括：
    在确定所述朝向表示第一人脸图像为正脸图像的情况下，基于所述第一脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第一中间点；
    基于所述第一中间点及所述眼尾关键点，确定所述目标区域。
  4. 根据权利要求2所述的方法，其特征在于，所述关键点包括眼尾关键点、第二脸型轮廓关键点和鼻翼关键点；
    所述基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域，包括：
    在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于所述第二脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第二中间点；
    基于所述第二中间点与所述眼尾关键点，确定所述目标区域。
  5. 根据权利要求2至4任一所述的方法，其特征在于，所述基于所述关键点，确定苹果肌的调整参数，包括：
    基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
  6. 根据权利要求5所述的方法，其特征在于，所述基于所述关键点及所述朝向，确定所述苹果肌的调整方向，包括：
    在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第一方向。
  7. 根据权利要求5所述的方法，其特征在于，所述基于所述关键点及所述朝向，确定所述苹果肌的调整方向，包括：
    在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
  8. 根据权利要求6所述的方法，其特征在于，所述第一方向为：所述目标区域的第一中间点指向眼尾关键点的方向。
  9. 根据权利要求7所述的方法，其特征在于，所述第二方向为：所述目标区域的第二中间点指向眼尾关键点的方向；所述第三方向为：鼻翼关键点指向确定目标区域的第二脸型轮廓关键点的方向。
  10. 根据权利要求2至4任一所述的方法，其特征在于，所述基于所述关键点，确定苹果肌的调整参数，包括：
    基于所述关键点和所述朝向，确定所述苹果肌的调整幅度。
  11. 根据权利要求10所述的方法，其特征在于，所述基于所述关键点和所述朝向，确定所述苹果肌的调整幅度，包括：
    基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离；
    在所述朝向表示所述第一人脸图像为侧脸图像的情况下，根据人脸偏转角及所述最大 移动距离，确定所述第一人脸图像的苹果肌的实际移动距离。
  12. 根据权利要求1至11任一项所述的方法，其特征在于，所述方法还包括：
    对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
  13. 根据权利要求12所述的方法，其特征在于，所述对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像，包括：
    从所述第二人脸图像中获取人脸的肤色参数；
    基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
    基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，获得所述第三人脸图像。
  14. 根据权利要求13所述的方法，其特征在于，所述对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像，包括：
    根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；
    基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
  15. 根据权利要求14所述的方法，其特征在于，所述根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像，包括：
    根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的所述掩码图像。
  16. 根据权利要求14或15所述的方法，其特征在于，所述基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像，包括：
    将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
    基于所述第二像素值，生成所述第三人脸图像。
  17. 根据权利要求16所述的方法，其特征在于，所述方法还包括：
    获取控制参数；
    所述基于所述第二像素值，生成所述第三人脸图像，包括：
    基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素值；
    基于所述第三像素值，生成所述第三人脸图像。
  18. 一种图像处理装置，其特征在于，包括：
    检测单元，用于检测第一人脸图像的关键点；
    第一确定单元，用于基于所述关键点，确定苹果肌调整的目标区域；
    第二确定单元，用于基于所述关键点，确定苹果肌的调整参数；
    调整单元，用于基于所述调整参数，调整所述目标区域形成第二人脸图像。
  19. 根据权利要求18所述的装置，其特征在于，所述装置还包括：
    第三确定单元，用于确定所述第一人脸图像中人脸的朝向；
    所述第一确定单元用于：
    基于所述关键点及所述朝向，确定苹果肌调整的所述目标区域。
  20. 根据权利要求19所述的装置，其特征在于，所述关键点包括：眼尾关键点、第一脸型轮廓关键点和鼻翼关键点；
    所述第二确定单元用于：
    在确定所述朝向表示第一人脸图像为正脸图像的情况下，基于所述第一脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第一中间点；
    基于所述第一中间点及所述眼尾关键点，确定所述目标区域。
  21. 根据权利要求19所述的装置，其特征在于，所述关键点包括眼尾关键点、第二脸型轮廓关键点和鼻翼关键点；
    所述第一确定单元用于：
    在确定所述朝向表示所述第一人脸图像为侧脸图像的情况下，基于所述第二脸型轮廓关键点和所述鼻翼关键点，确定所述目标区域的第二中间点；
    基于所述第二中间点与所述眼尾关键点，确定所述目标区域。
  22. 根据权利要求19至21任一所述的装置，其特征在于，所述第二确定单元用于：
    基于所述关键点及所述朝向，确定所述苹果肌的调整方向。
  23. 根据权利要求22所述的装置，其特征在于，所述第三确定单元用于：
    在所述朝向表示第一人脸图像为正脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第一方向。
  24. 根据权利要求22所述的装置，其特征在于，所述第三确定单元还用于：
    在所述朝向表示第一人脸图像为侧脸图像的情况下，基于所述关键点确定苹果肌的调整方向为第二方向和第三方向，其中，所述第三方向不同于所述第二方向。
  25. 根据权利要求23所述的装置，其特征在于，所述第一方向为：所述目标区域的第一中间点指向眼尾关键点的方向。
  26. 根据权利要求24所述的装置，其特征在于，所述第二方向为：所述目标区域的第二中间点指向眼尾关键点的方向；所述第三方向为：鼻翼关键点指向确定目标区域的第二脸型轮廓关键点的方向。
  27. 根据权利要求19至22任一所述的装置，其特征在于，所述第二确定单元用于：
    基于所述关键点和所述朝向，确定所述苹果肌的调整幅度。
  28. 根据权利要求27所述的装置，其特征在于，所述第二确定单元用于：
    基于眼尾关键点和眼头关键点，确定所述苹果肌的最大移动距离；
    在所述朝向表示所述第一人脸图像为侧脸图像的情况下，根据人脸偏转角及所述最大移动距离，确定所述第一人脸图像的苹果肌的实际移动距离。
  29. 根据权利要求18至28任一项所述的装置，其特征在于，所述装置还包括：
    提亮处理单元，用于对所述第二人脸图像中的苹果肌进行提亮处理，获得第三人脸图像。
  30. 根据权利要求29所述的装置，其特征在于，所述提亮处理单元用于：
    从所述第二人脸图像中获取人脸的肤色参数；
    基于所述人脸肤色参数，确定苹果肌提亮处理的提亮参数；
    基于所述提亮参数对所述第二人脸图像中的苹果肌进行提亮处理，获得所述第三人脸图像。
  31. 根据权利要求30所述的装置，其特征在于，所述提亮处理单元用于：
    根据所述第二人脸图像中苹果肌的所在位置，生成掩码图像；
    基于所述掩码图像，对所述第二人脸图像进行所述苹果肌的提亮处理得到所述第三人脸图像。
  32. 根据权利要求31所述的装置，其特征在于，所述提亮处理单元用于：
    根据预先设定的苹果肌提亮的亮斑，及所述第二人脸图像中苹果肌所在的位置，生成包含有所述亮斑的所述掩码图像。
  33. 根据权利要求31或32所述的装置，其特征在于，所述提亮处理单元用于：
    将所述掩码图像中像素值大于预定阈值的像素，与所述第二人脸图像中对应位置像素的第一像素值进行混合，得到混合后的第二像素值；
    基于所述第二像素值，生成所述第三人脸图像。
  34. 根据权利要求33所述的装置，其特征在于，所述装置还包括：
    获取单元，用于获取控制参数；
    所述提亮处理单元用于：
    基于所述控制参数、所述第二像素值和所述第一像素值进行线性混合，得到第三像素 值；
    基于所述第三像素值，生成所述第三人脸图像。
  35. 一种图像设备，其特征在于，包括：
    存储器；
    处理器，与所述存储器连接，用于通过执行存储在所述存储器上的计算机可执行指令，实现权利要求1至17任一项提供的方法。
  36. 一种计算机存储介质，所述计算机存储介质存储有计算机可执行指令；所述计算机可执行指令被执行后，能够实现权利要求1至17任一项提供的方法。
PCT/CN2019/124515 2019-03-06 2019-12-11 图像处理方法及装置、图像设备及存储介质 Ceased WO2020177434A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2020535619A JP6990773B2 (ja) 2019-03-06 2019-12-11 画像処理方法及び装置、画像デバイス並びに記憶媒体
KR1020207019432A KR102386642B1 (ko) 2019-03-06 2019-12-11 이미지 처리 방법 및 장치, 전자 기기 및 저장 매체
SG11202006333VA SG11202006333VA (en) 2019-03-06 2019-12-11 Image processing method and apparatus, image device, and storage medium
US16/914,637 US11238569B2 (en) 2019-03-06 2020-06-29 Image processing method and apparatus, image device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910169460.XA CN109949237A (zh) 2019-03-06 2019-03-06 图像处理方法及装置、图像设备及存储介质
CN201910169460.X 2019-03-06

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/914,637 Continuation US11238569B2 (en) 2019-03-06 2020-06-29 Image processing method and apparatus, image device, and storage medium

Publications (1)

Publication Number Publication Date
WO2020177434A1 true WO2020177434A1 (zh) 2020-09-10

Family

ID=67009236

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/124515 Ceased WO2020177434A1 (zh) 2019-03-06 2019-12-11 图像处理方法及装置、图像设备及存储介质

Country Status (7)

Country Link
US (1) US11238569B2 (zh)
JP (1) JP6990773B2 (zh)
KR (1) KR102386642B1 (zh)
CN (1) CN109949237A (zh)
SG (1) SG11202006333VA (zh)
TW (1) TWI752419B (zh)
WO (1) WO2020177434A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112906512A (zh) * 2021-02-03 2021-06-04 北京海迩西医疗科技有限公司 一种确定人体关节的方法、装置以及存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109949237A (zh) 2019-03-06 2019-06-28 北京市商汤科技开发有限公司 图像处理方法及装置、图像设备及存储介质
CN111754415B (zh) 2019-08-28 2022-09-27 北京市商汤科技开发有限公司 人脸图像处理方法及装置、图像设备及存储介质
CN110555896B (zh) * 2019-09-05 2022-12-09 腾讯科技（深圳）有限公司 一种图像生成方法、装置以及存储介质
CN111680546A (zh) * 2020-04-26 2020-09-18 北京三快在线科技有限公司 注意力检测方法、装置、电子设备及存储介质
CN112001872B (zh) * 2020-08-26 2021-09-14 北京字节跳动网络技术有限公司 信息显示方法、设备及存储介质
CN112767288B (zh) * 2021-03-19 2023-05-12 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN113674139B (zh) * 2021-08-17 2024-08-20 北京京东尚科信息技术有限公司 人脸图像的处理方法、装置、电子设备及存储介质
CN118115576A (zh) * 2022-11-30 2024-05-31 华为技术有限公司 图像处理方法、装置及相关设备
TWI831582B (zh) * 2023-01-18 2024-02-01 瑞昱半導體股份有限公司 偵測系統以及偵測方法
CN116596786A (zh) * 2023-05-15 2023-08-15 微智医疗器械有限公司 用于视觉假体的图像处理方法、视觉假体和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109376684A (zh) * 2018-11-13 2019-02-22 广州市百果园信息技术有限公司 一种人脸关键点检测方法、装置、计算机设备和存储介质
CN109376671A (zh) * 2018-10-30 2019-02-22 北京市商汤科技开发有限公司 图像处理方法、电子设备及计算机可读介质
CN109377446A (zh) * 2018-10-25 2019-02-22 北京市商汤科技开发有限公司 人脸图像的处理方法及装置、电子设备和存储介质
CN109949237A (zh) * 2019-03-06 2019-06-28 北京市商汤科技开发有限公司 图像处理方法及装置、图像设备及存储介质

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20040075672A (ko) * 2003-02-21 2004-08-30 정보통신연구진흥원 계층적 가상 성형시뮬레이션 방법
JP2004318204A (ja) 2003-04-10 2004-11-11 Sony Corp 画像処理装置、画像処理方法及び撮影装置
JP2007193730A (ja) * 2006-01-23 2007-08-02 Seiko Epson Corp 印刷装置、画像処理装置、印刷方法、および画像処理方法
JP2007193729A (ja) * 2006-01-23 2007-08-02 Seiko Epson Corp 印刷装置、画像処理装置、印刷方法、および画像処理方法
JP2009064423A (ja) * 2007-08-10 2009-03-26 Shiseido Co Ltd メイクアップシミュレーションシステム、メイクアップシミュレーション装置、メイクアップシミュレーション方法およびメイクアップシミュレーションプログラム
JP2009211513A (ja) * 2008-03-05 2009-09-17 Toshiba Corp 画像処理装置及びその方法
CN103236066A (zh) * 2013-05-10 2013-08-07 苏州华漫信息服务有限公司 一种基于人脸特征分析的虚拟试妆方法
CN106991367B (zh) * 2016-01-21 2019-03-19 腾讯科技（深圳）有限公司 确定人脸转动角度的方法和装置
CN107103271A (zh) * 2016-02-23 2017-08-29 芋头科技（杭州）有限公司 一种人脸检测方法
CN107305622B (zh) * 2016-04-15 2021-08-31 北京市商汤科技开发有限公司 一种人脸五官识别方法、装置及系统
CN108734070A (zh) * 2017-04-24 2018-11-02 丽宝大数据股份有限公司 腮红指引装置及方法
KR20200014280A (ko) * 2017-05-31 2020-02-10 소니 주식회사 화상 처리 장치, 화상 처리 시스템, 및 화상 처리 방법, 그리고 프로그램
CN108230331A (zh) * 2017-09-30 2018-06-29 深圳市商汤科技有限公司 图像处理方法和装置、电子设备、计算机存储介质
CN108876731A (zh) * 2018-05-25 2018-11-23 北京小米移动软件有限公司 图片处理方法及装置
CN108986019A (zh) * 2018-07-13 2018-12-11 北京小米智能科技有限公司 肤色调整方法及装置、电子设备、机器可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109377446A (zh) * 2018-10-25 2019-02-22 北京市商汤科技开发有限公司 人脸图像的处理方法及装置、电子设备和存储介质
CN109376671A (zh) * 2018-10-30 2019-02-22 北京市商汤科技开发有限公司 图像处理方法、电子设备及计算机可读介质
CN109376684A (zh) * 2018-11-13 2019-02-22 广州市百果园信息技术有限公司 一种人脸关键点检测方法、装置、计算机设备和存储介质
CN109949237A (zh) * 2019-03-06 2019-06-28 北京市商汤科技开发有限公司 图像处理方法及装置、图像设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112906512A (zh) * 2021-02-03 2021-06-04 北京海迩西医疗科技有限公司 一种确定人体关节的方法、装置以及存储介质
CN112906512B (zh) * 2021-02-03 2024-06-11 北京海迩西医疗科技有限公司 一种确定人体关节的方法、装置以及存储介质

Also Published As

Publication number Publication date
CN109949237A (zh) 2019-06-28
US11238569B2 (en) 2022-02-01
SG11202006333VA (en) 2020-10-29
TWI752419B (zh) 2022-01-11
KR20200107957A (ko) 2020-09-16
KR102386642B1 (ko) 2022-04-14
JP6990773B2 (ja) 2022-01-12
JP2021517676A (ja) 2021-07-26
TW202036467A (zh) 2020-10-01
US20200327647A1 (en) 2020-10-15

Similar Documents

Publication Publication Date Title
KR102386642B1 (ko) 이미지 처리 방법 및 장치, 전자 기기 및 저장 매체
JP6864449B2 (ja) イメージの明るさを調整する方法及び装置
US11250241B2 (en) Face image processing methods and apparatuses, and electronic devices
US10739849B2 (en) Selective peripheral vision filtering in a foveated rendering system
US9639914B2 (en) Portrait deformation method and apparatus
WO2022078041A1 (zh) 遮挡检测模型的训练方法及人脸图像的美化处理方法
US9135726B2 (en) Image generation apparatus, image generation method, and recording medium
CN111339928B (zh) 眼神调节方法、装置及存储介质
CN110688948B (zh) 视频中人脸性别变换方法、装置、电子设备和存储介质
CN106920274A (zh) 移动端2d关键点快速转换为3d融合变形的人脸建模方法
KR20180066160A (ko) 안면 이미지 처리 방법 및 장치, 및 저장 매체
CN113344837B (zh) 人脸图像处理方法及装置、计算机可读存储介质、终端
CN110533773A (zh) 一种三维人脸重建方法、装置及相关设备
CN114187166A (zh) 图像处理方法、智能终端及存储介质
CN114841853A (zh) 图像处理方法、装置、设备及存储介质
CN110852934A (zh) 图像处理方法及装置、图像设备及存储介质
WO2025241628A1 (zh) 图像与视频处理方法、设备、存储介质及程序产品
CN112562066A (zh) 图像重建方法和装置、电子设备
CN114140580B (zh) 一种手部三维模型的纹理调节方法及设备
JP6376673B2 (ja) 画像処理装置
HK40010064A (zh) 图像处理方法及装置、图像设备及存储介质
CN112102374B (zh) 图像处理方法、装置、电子设备及介质
CN121982271A (zh) 图像处理方法及装置

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2020535619

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19918192

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19918192

Country of ref document: EP

Kind code of ref document: A1