KR102386642B1 - Image processing method and apparatus, electronic device and storage medium - Google Patents

Abstract

Embodiments of the present invention disclose an image processing method and apparatus, an image device, and a storage medium. The image processing method may include: detecting a key point of a first face image; determining a target area adjusted by the Apple Zone based on the key point; determining an adjustment parameter of the Apple Zone based on the key point; and adjusting the target area based on the adjustment parameter to form a second face image.

Description

Image processing method and apparatus, electronic device and storage medium

Cross-referencing of related applications

This application is filed with the Chinese Intellectual Property Office on March 6, 2019, the application number is 201910169460.X, and the title of the invention is “Image processing method and apparatus, image device and storage medium”, the entire contents of which are hereby incorporated by reference bound to the application.

The present application relates to the field of computer technology, and more particularly, to an image processing method, an image processing apparatus, an image device, and a storage medium.

Cell phones, tablet computers, or wearables, etc. are usually equipped with cameras that can collect images. However, the collected image need not be the ideal image desired by the user, and may need to be adjusted to meet the user's expectations, for example to make the image beautiful, comical, or cute, and may need to be adapted to the processing of a person, for example. Regarding image processing in technology, it is generally limited to processing of eyes, nose and face shapes, but processing such as beautification of other parts is relatively small. Therefore, the image processing technology of the related art is not optimized and not perfect, and the processing effect of the image has not reached the ideal expected effect.

In consideration of this, embodiments of the present invention provide an image processing method and apparatus, an image device, and a storage medium.

In a first aspect, there is provided an image processing method, the image processing method comprising:

detecting a key point of the first face image;

determining a target area adjusted by the Apple Zone based on the key point;

determining an adjustment parameter of the Apple Zone based on the key point; and

and adjusting the target area based on the adjustment parameter to form a second face image.

In combination with any embodiment of the present invention, before determining, based on the key point, the target area adjusted by the applezone, the image processing method comprises:

determining the orientation of the face in the first face image;

Determining the target area adjusted by the Apple Zone based on the key point comprises:

and determining, based on the key point and the direction, the target area adjusted by the applezone.

Combined with any of the embodiments of the present invention, the key points include an eye corner key point, a first facial contour key point and a nostril key point;

Determining the target area adjusted by the Apple Zone based on the key point and the direction comprises:

when it is determined that the direction indicates that the first facial image is a frontal facial image, determining a first midpoint of the target area based on the first facial contour key point and the nostril key point; and

and determining the target area based on the first midpoint and the key point of the corner of the eye.

In combination with any of the embodiments of the present invention, the key points include an eye corner key point, a second facial contour key point and a nostril key point;

Determining the target area adjusted by the Apple Zone based on the key point and the direction comprises:

when it is determined that the direction indicates that the first face image is a side face image, determining a second midpoint of the target area based on the second facial contour key point and the nostril key point; and

and determining the target area based on the second midpoint and the key point of the corner of the eye.

In combination with any embodiment of the present invention, the step of determining, based on the key point, an adjustment parameter of the apple zone,

and determining an adjustment direction of the apple zone based on the key point and the direction.

When combined with any embodiment of the present invention, the step of determining the adjustment direction of the apple zone based on the key point and the direction comprises:

and when the direction indicates that the first face image is a front face image, determining that the adjustment direction of the Apple Zone is the first direction based on the key point.

When combined with any embodiment of the present invention, the step of determining the adjustment direction of the apple zone based on the key point and the direction comprises:

when the direction indicates that the first face image is a side face image, determining that the adjustment directions of the apple zone are a second direction and a third direction based on the key point, wherein the third direction is different from the second direction includes -.

Combined with any embodiment of the present invention, the first direction is a direction in which the first midpoint of the target area points to the corner key point.

Combined with any of the embodiments of the present invention, the second direction is a direction in which a second midpoint of the target area points to a corner key point;

The third direction is a direction in which the nose bridge key point points to the second facial contour key point determining the target area.

In combination with any embodiment of the present invention, the step of determining, based on the key point, an adjustment parameter of the apple zone,

and determining an adjustment width of the apple zone based on the key point and the direction.

In combination with any embodiment of the present invention, the step of determining the adjustment width of the apple zone based on the key point and the direction comprises:

determining the maximum moving distance of the apple zone based on the key point of the corner of the eye and the key point of the corner of the eye; and

and when the direction indicates that the first face image is a side face image, determining an actual moving distance of the apple zone of the first face image according to a face deflection angle and the maximum moving distance.

Combined with any embodiment of the present invention, the image processing method comprises:

The method further includes performing a brightening process on the apple zone in the second face image to obtain a third face image.

In combination with any embodiment of the present invention, the step of performing a brightening process on the apple zone in the second face image to obtain a third face image includes:

obtaining a skin color parameter of the face from the second face image;

determining a brightening parameter for apple zone brightening treatment based on the facial skin color parameter; and

and performing a brightening process on the apple zone in the second face image based on the brightening parameter to obtain the third face image.

In combination with any embodiment of the present invention, the step of performing a brightening process on the apple zone in the second face image to obtain a third face image includes:

generating a mask image according to the position of the apple zone in the second face image; and

and obtaining the third face image by performing the Apple Zone brightening process on the second face image based on the mask image.

In combination with any embodiment of the present invention, according to the position of the apple zone in the second face image, generating a mask image includes:

and generating the mask image including the bright point according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image.

In combination with any embodiment of the present invention, based on the mask image, performing the apple zone brightening process on the second face image to obtain the third face image,

obtaining a mixed second pixel value by mixing a pixel of the mask image whose pixel value is greater than a predetermined threshold with a first pixel value of a position pixel corresponding to the second face image; and

and generating the third face image based on the second pixel value.

Combined with any embodiment of the present invention, the image processing method comprises:

further comprising obtaining a control parameter;

Based on the second pixel value, generating the third face image includes:

performing linear mixing based on the control parameter, the second pixel value, and the first pixel value to obtain a third pixel value; and

and generating the third face image based on the third pixel value.

In a second aspect, there is provided an image processing device,

a detection unit for detecting a key point of the first face image;

a first determining unit for determining, based on the key point, a target area adjusted by the apple zone;

a second determining unit for determining, based on the key point, an adjustment parameter of the apple zone; and

and an adjustment unit, configured to adjust the target area to form a second face image, based on the adjustment parameter.

Combined with any embodiment of the present invention, the image processing apparatus comprises:

a third determining unit for determining a direction of a face in the first face image;

The first determining unit is

to determine the target area adjusted by the applezone based on the key point and the direction.

Combined with any of the embodiments of the present invention, the key points include an eye corner key point, a first facial contour key point and a nostril key point;

the second determining unit,

when it is determined that the direction indicates that the first facial image is a frontal facial image, determine a first midpoint of the target area based on the first facial contour key point and the nostril key point;

Based on the first midpoint and the key point of the corner of the eye, the target area is determined.

In combination with any of the embodiments of the present invention, the key points include an eye corner key point, a second facial contour key point and a nostril key point;

The first determining unit is

if it is determined that the direction indicates that the first face image is a side face image, determine a second midpoint of the target area based on the second facial contour key point and the nostril key point;

Based on the second midpoint and the key point of the corner of the eye, the target area is determined.

When combined with any embodiment of the present invention, the second determining unit comprises:

Based on the key point and the direction, to determine the adjustment direction of the apple zone.

When combined with any embodiment of the present invention, the third determining unit comprises:

When the direction indicates that the first face image is a front face image, it is determined that the adjustment direction of the Apple Zone is the first direction based on the key point.

Combined with any embodiment of the present invention, the third determining unit further comprises:

when the direction indicates that the first face image is a side face image, determining that the adjustment directions of the apple zone are a second direction and a third direction based on the key point, wherein the third direction is the second direction different from the direction

Combined with any embodiment of the present invention, the first direction is a direction in which the first midpoint of the target area points to the corner key point.

Combined with any of the embodiments of the present invention, the second direction is a direction in which a second midpoint of the target area points to a corner key point;

The third direction is a direction in which the nose bridge key point points to the second facial contour key point determining the target area.

When combined with any embodiment of the present invention, the second determining unit comprises:

based on the key point and the direction, to determine an adjustment width of the apple zone.

When combined with any embodiment of the present invention, the second determining unit comprises:

determining the maximum moving distance of the apple zone based on the key points of the corners of the eyes and the key points of the corners of the eyes;

When the direction indicates that the first face image is a side face image, an actual moving distance of the apple zone of the first face image is determined according to a face deflection angle and the maximum moving distance.

Combined with any embodiment of the present invention, the image processing apparatus comprises:

and a brightening processing unit configured to perform brightening processing on the apple zone in the second facial image to obtain a third facial image.

Combined with any embodiment of the present invention, the brightening processing unit comprises:

obtain a skin color parameter of the face from the second face image;

determine a brightening parameter for apple zone brightening treatment based on the facial skin color parameter;

Brightening processing is performed on the apple zone in the second face image based on the brightening parameter to obtain the third face image.

Combined with any embodiment of the present invention, the brightening processing unit comprises:

generate a mask image according to the position of the apple zone in the second face image;

Based on the mask image, the Apple Zone brightening process is performed on the second face image to obtain the third face image.

Combined with any embodiment of the present invention, the brightening processing unit comprises:

This is to generate the mask image including the bright point according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image.

Combined with any embodiment of the present invention, the brightening processing unit comprises:

a pixel having a pixel value greater than a predetermined threshold value of the mask image is mixed with a first pixel value of a position pixel corresponding to the second face image to obtain a mixed second pixel value;

Based on the second pixel value, the third face image is generated.

Combined with any embodiment of the present invention, the image processing apparatus comprises:

an acquiring unit for acquiring the control parameter;

The brightening processing unit,

perform linear mixing based on the control parameter, the second pixel value, and the first pixel value to obtain a third pixel value;

Based on the third pixel value, the third face image is generated.

In a third aspect, the provided imaging device comprises:

Memory; and

and a processor coupled to the memory and configured to execute computer-executable instructions stored in the memory to implement the first aspect and any embodiments thereof.

A computer storage medium having computer-executable instructions stored therein, and after the computer-executable instructions are executed, the above-described first aspect and any embodiments thereof can be implemented.

The technical solution provided in the embodiment of the present invention is to perform key point detection on the first face image to obtain a key point in the first face image, and then, based on the key point, perform key point detection in the first face image. If the target area where the zone is located is determined, and an adjustment parameter for adjusting the apple zone of the first face image is determined based on the key point, and the target area is adjusted using the adjustment parameter, the second target area after the apple zone is adjusted is determined. 2 You can create a face image. In this way, the technical solution provided in the embodiment of the present invention can at least provide the function of adjusting the apple zone in the face image, and subsequently adjust the apple zone in the face image according to the user's demand, Enhances beautifying and comical effects for face images.

In order to more clearly explain the technical solutions in the embodiments or the background art of the present application, the drawings necessary to explain the embodiments or the background art of the present application will be described.
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are incorporated in and constitute a part of this specification, and the accompanying drawings show an embodiment consistent with the present invention, and together with the specification are used to describe the technical solution of the present invention.
1 is a schematic diagram of a pixel coordinate system provided in an embodiment of the present invention.
2 is a schematic diagram of region division of a face image provided in an embodiment of the present invention.
3 is a flow schematic diagram of an image processing method provided in an embodiment of the present invention.
4 is a schematic diagram of face key points provided in an embodiment of the present invention.
5 is a flowchart of another image processing method provided in an embodiment of the present invention.
6A is a schematic diagram of the effect of a front face person provided in an embodiment of the present invention.
6B is a schematic diagram of the effect of a side face person provided in an embodiment of the present invention.
6C is a schematic diagram of the effect of another side face person provided in an embodiment of the present invention.
7 is a schematic diagram of the effect of a mask image provided in an embodiment of the present invention.
8 is a structural schematic diagram of an image processing apparatus provided in an embodiment of the present invention.
9 is a structural schematic diagram of an image providing apparatus according to an embodiment of the present invention.
10 is a schematic diagram of the effect of the image processing method provided in the embodiment of the present invention.

In combination with the drawings and specific embodiments of the present specification below, the technical method of the present invention will be described in detail.

Prior to the following description, the pixel coordinate system in the embodiment of the present invention is first defined. 1, the lower right corner of the face image (A) is used as the origin (o) of the pixel coordinate system, the direction parallel to the row of the face image (A) is the x-axis direction, and the face image (A) ), the direction parallel to the column is taken as the y-axis direction, thereby constituting the pixel coordinate system (xoy). In the pixel coordinate system, the abscissa is for representing the number of columns of pixels in the face image (A) in the face image (A), and the ordinate represents the number of rows of pixels in the face image (A) in the face image (A). For the purpose of betting, both the abscissa and ordinate units may be pixels. For example, if the coordinates of pixel a in Fig. 1 are (10, 30), that is, the abscissa of pixel a is 10 pixels, the ordinate of pixel a is 30 pixels, and pixel a is a face image (A ) in column 10, row 30.

For convenience of description, an embodiment of the present invention divides a face region in a face image into a left face region and a right face region. As shown in FIG. 2 , the face region is divided into a left face region and a right face region based on a symmetric center line of the face region.

As shown in Fig. 3, the image processing method provided in this embodiment includes the following method.

In step S110, a key point of the first face image is detected.

In step S120, a target area adjusted by the Apple Zone is determined based on the key point.

In step S130, an adjustment parameter of the Apple Zone is determined based on the key point.

In step S140, a second face image is formed by adjusting the target area based on the adjustment parameter.

In the present embodiment, the apple zone on the face may be understood as an inverted triangular tissue region located in the area under the eyes. This area swells slightly due to the compression of facial muscles when a person smiles or makes an expression, and the area in which the inverted triangle tissue is located looks like a round and glossy apple, so it is called Apple Zone. This embodiment provides an adjustment method for the Apple Zone, and in actual application, it can also be applied to other parts of a human face, a face part of a cartoon character, an animal face part, etc., based on the technical concept of the adjustment scheme, the face Adjusted for the site, it can be considered to be within the technical scope of the embodiment of the present invention.

In step S110, a face key point in the first face image is determined by performing processing on the first face image using a deep learning model such as a neural network or a face key point detection algorithm. The face key point detection algorithm may include OpenFace, multi-task cascaded convolutional networks (MTCNN), tweaked convolutional neural networks (TCNN), or task-limited deep convolutional networks. (tasks-constrained deep convolutional network, TCDCN), and the present invention is not limited to the face key point detection algorithm.

For example, the key point is

face contour key points, which are key points 0 to 32 shown in Fig. 4;

eyebrow key points that are key point 33 to key point 37, key point 38 to key point 42, and key point 64 to key point 71 shown in FIG. 4;

eye key point;

foot key point; and

It includes, but is not limited to, key points such as lip key points, and the lip key points may refer to key points 84 to 103 of FIG. 4 .

wherein the eye key points depend on the distribution of key points in the eye;

key points located in the inner canthus, such as key point 55 and key point 58 shown in FIG. 4;

a key point located at the lateral canthus, such as key point 52 and key point 62 shown in FIG. 4;

upper eyelid key points located on the upper eyelid, such as key points 53, 72 and 54, and key points 59, 75 and 60 shown in FIG. 4;

the lower eyelid key points located on the lower eyelid, such as key points 62, 76 and 63, and key points 57, 73 and 56 shown in FIG. 4; and

may include ocular key points located in the eye, such as key points 74 and 104 and key points 77 and 105 shown in FIG. 4 ;

The nose key point is according to the height distribution in the nose,

a key point located at a mountain root, such as the key point 43 shown in FIG. 4 ;

a nasal key point located on the nose, such as key point 44 and key point 45 in FIG. 4;

a nostril key point located on the nostril, such as key point 80 through key point 83 in FIG. 4;

a nose tip key point located at the tip of the nose, such as the key point 46 of FIG. 4; and

It can be divided into key points located under the nose, such as the key point 47 to the key point 51 shown in FIG. 4 .

In this embodiment, the key points related to Apple Zone adjustment are:

a key point 4, a key point 5, a key point 28, and a key point 27 at positions close to the cheekbones in the facial contour key points shown in Fig. 4;

eyebrow key point; and

It includes at least one of the eyebrow key points.

In step S310, a target area for performing apple zone adjustment in the first face image is displayed according to one or a plurality of key points in the key point. After displaying the area, adjustment of the apple zone may be implemented based on the pixels in the area. In a specific embodiment, since at least one of the first facial contour key point and the second facial contour key point can be selected as the facial contour key point, an adjustment parameter of the Apple Zone is determined.

In this embodiment, the target area may be a circular area or an elliptical area.

In step S310, according to one or a plurality of key points at the key point, an adjustment parameter of the Apple Zone is determined. The adjustment parameters of the Apple Zone are,

Adjustment direction of Apple Zone;

Apple Zone adjustment width; and

At least one of the gradients of the Apple Zone includes, but is not limited to.

the adjustment direction may be used to determine a movement direction of the applezone in the first facial image;

The adjustment width may be used to determine a change measure such as a moving distance or a scale ratio of the apple zone in the first face image;

The adjustment gradient can be used to adjust the adjustment width difference between different adjustments when adjusted multiple times, or the adjustment width difference between different areas when the adjustment is performed on a divided area of the Apple Zone. dividing, for example, a circular or elliptical target area into an inner area and an outer area; The outer area is surrounded by the inner area, and when brightness adjustment is performed for the Apple Zone, if the brightness adjustment width of the inner area is larger than the brightness adjustment width of the outer area, the difference in the adjusted brightness between the inner area and the outer area is Make up gradients. Of course, this is just an example, and there are a plurality of specific implementations.

In step S310, the adjustment parameter and the key point for determining the target area may be the same or different. In some embodiments, a portion of the key point for determining the adjustment parameter and the target area may be the same. For example, the same facial contour key point may be used to determine the adjustment parameter and the target area. Again, for example, the same eye key point may be used simultaneously to determine the adjustment parameter and the target area.

In step S140 , the second face image is formed by adjusting the target area of at least one of a position, a shape, a dimension, or a color in the first face image according to the adjustment parameter.

Specifically, the step S140 is

adjusting a position of the target area by moving the target area;

adjusting a dimension of the target area in a second face image by scaling the target area;

forming a highlight area or a shadow area of a face by performing color conversion on the target area and adjusting the color of the target area; and

and performing a shape change in the target area to adjust the shape shown in the target area, thereby imaging an apple zone shape suitable for a facial shape of the face. For example, different people image different face shapes, for example, some oval faces, some heart-shaped faces, some round faces, some diamond faces. The shape of the Apple Zone is generally an inverted triangle; If the angles or vertices of the three corners of the inverted triangle are different from each other in the face image, it makes Apple Zone show different visual impressions. Thus, in some embodiments, the imaging device may also adjust the position, etc. of pixels within the target area, so as to match the face shape, so that the Apple Zone can be adapted to fit different shaped face shapes within the target area or within the target area after movement. By making it appear as a triangle, it is possible to implement processing such as beautifying a person or a comic.

In short, the method provided in this embodiment can implement the Apple Zone of a beautiful, comical, cute or different style type of Apple Zone in the face image by performing the adjustment on the Apple Zone in the face image; In this way, it is possible to implement apple zone processing of images according to user requirements, thereby satisfying different image needs of users, and improving user experience and image processing quality.

In some embodiments, when the adjustment parameter is generated in step S130, the adjustment parameter may be generated according to the expected adjustment effect of the current Apple Zone. For example, if the currently expected adjustment effect is the Apple Zone beautifying effect, according to the beautifying effect, Apple Zone exhibits a relatively popular aesthetic beautifying effect in the second face image than in the first face image. Again, for example, if the currently expected adjustment effect is a comic effect, a humor effect is expressed through adjustment of the apple zone in the first face image; When the adjustment parameter is generated, the key point and the comic effect are combined to generate an apple zone in which a second face image exhibits a humorous gag effect. In sum, in some embodiments, the generation of the adjustment parameter is also determined according to the expected adjustment effect.

In this embodiment, the adjustment for the Apple Zone can be used in the image beautification process, can be used in a beauty application, and the Apple Zone adjustment provided in the embodiment of the present invention can be used for one-click portrait beauty, in this way , Apple Zone beautification is implemented as one function of one-click portrait beauty along with beautification of eyes, nose, or face shape.

Of course, the adjustment of the Apple Zone provided in this embodiment can also be used for a professional Apple Zone beautification function. For example, when the Apple Zone beautification control is set in the beauty application, and a user's manipulation acting on the Apple Zone beautification control is detected, the Apple Zone in the image is independently beautified using the above method.

In short, the image processing method described in this embodiment has an Apple Zone adjustment function and can perform adjustments to the Apple Zone in the process of performing person processing, thereby satisfying the user's needs for Apple Zone adjustment and , improve user experience and image device intelligence.

In some other embodiments, as shown in FIG. 5 , the image processing method includes the following steps.

In step S111, the direction of the face in the first face image is determined.

The step S120 may include step S121; The step S121 may include determining the target area adjusted by the Apple Zone based on the key point and the direction.

In this embodiment, it also determines the direction of the first face image. The direction of the first face image is divided into at least two directions.

in the first direction, the front face direction, that is, the first face image is indicated as the front face image;

In the second direction, it is indicated that the side face direction, that is, the first face image is the side face image.

When an included angle (referred to as a face deflection angle below) between a photographing direction of an imaging device that collects the first face image and a symmetric center line of a face region of a person to be photographed is located within a first preset range, the first face The orientation of the image is the front face orientation.

The face deflection angle refers to an angle between the photographing direction of the imaging device and the vertical line of the face region of the person being photographed. When clockwise with respect to the offset direction of the symmetrical centerline, the face deflection angle is positive; If it is counter-clockwise to the direction, the face deflection angle is negative.

When the face deflection angle of the first face image is within a second preset range, the direction of the first face image is a side face direction.

The first preset range and the second preset range are sets complementary to each other. For example, if the first preset range is -10 degrees to 10 degrees, the second preset range is -180 degrees to -10 degrees, and 10 degrees to 180 degrees.

All directions other than the front face direction in the face image may be regarded as side face directions, and of course, in this case, the back image not including the face is excluded.

6A is a face image in the front face direction; 6B is a face image in a lateral face direction; 6C is a face image in another lateral face direction.

In some embodiments, if the facial deflection angle of the facial image shown in FIG. 6A is 0 degrees and the facial deflection angle of the face image shown in FIG. 6B is -90 degrees, the facial deflection angle of the face image shown in FIG. 6C is It is 90 degrees.

Of course, the first preset range and the second preset range may be set as needed.

The area covered by the apple zone in the face image in the front face direction (hereinafter referred to as the front face image) and the area covered by the apple zone in the face image in the side face direction (hereinafter referred to as the side face image) are When using the same adjustment standard or adjustment parameters for the front face image and the side face image because of the different (including the area covered by the Apple Zone being different, the angle of the area covered by the Apple Zone being different, etc.) , the effect of adjusting the apple zone in the face image is poor (for example, the ratio of the area of the apple zone area to the area of the face area after adjustment is inappropriate, for example, the color and face area of the adjusted apple zone area) In the non-apple zone, the color difference is large). To this end, the embodiment of the present invention determines the adjustment parameters using different methods for face images in different directions, so as to improve the adjustment effect of the Apple Zone on the face images.

Hereinafter, the similarities and differences between the front face image and the side face image when determining the target area will be introduced respectively.

In the case of a front face image, the step S120 is

when it is determined that the direction indicates that the first facial image is a frontal facial image, determining a first midpoint of the target region based on a first facial contour key point and the nostril key point .

The first facial contour key point may be a facial contour key point of vertical coordinates within a first target range, wherein the first target range is the sum of the vertical coordinate range of the area covered by the Apple Zone and the third preset range. am. Optionally, the third preset range is from -5 pixels to 5 pixels. For example, if the maximum value of the vertical coordinate of a pixel in the area covered by the Apple Zone is 35 pixels, the minimum value is 16 pixels, and the third preset range is -4 to 5 pixels, the vertical coordinate is 16 A facial contour key point greater than or equal to -4=12 pixels and less than or equal to 35+5=40 pixels is a first facial contour key point.

Optionally, referring to FIG. 4 , key point 4 and key point 5 may be used as first facial contour key points for adjusting the left apple zone (ie, the apple zone located in the left face region) in the face image.

In this embodiment, preferably key point 5 is used as the first facial contour key point. The nostril key point is a key point in which a difference value between the vertical coordinates and the vertical coordinates of the first facial contour key point is located within a fourth preset range, and is located within the nostril area (the area is defined according to user requirements). For example, if the maximum value of the ordinate of the key point in the first facial contour key point is 50 pixels, the minimum value is 40 pixels, and the fourth preset range is 10 pixels, the ordinate of the key point of the nostril is 30 Nostril key points greater than or equal to n pixels and less than or equal to 60 pixels.

Optionally, the key point of the nostril may be the key point 80 or the key point 82 shown in FIG. 4 , and preferably, the key point 80 is selected as the key point of the nostril in this embodiment. Optionally, if the first face image is a front face image, the key point 5 and the key point 80 shown in FIG. 4 for adjusting the right apple zone (that is, located in the apple zone of the right face region) respectively It is selected as the first facial contour key point and nostril key point, and the adjustment effect further satisfies user expectations. The key point 27 and the key point 81 shown in Fig. 4 are selected as the first facial contour key point and the nostril key point of the left facial apple zone, respectively.

The first midpoint is the midpoint between the first facial contour key point and the nose bridge key point (the midpoint between the key point 80 and the key point 82 shown in Fig. 4, the key point 81 and the key midpoints between points 83).

In some embodiments, if the facial features of the facial image represented by the first facial image are different, position correction is performed based on the midpoint of the first facial contour key point and the nostril key point, and then the first midpoint is determined acquire For example, if the first face image represents the face A, it is assumed that the distance between the first facial contour key point from the eye key point to the corresponding position is the first distance, and the nostril at the corresponding position at the eye key point is assumed to be the first distance. It is assumed that the distance to the midpoint of the key point is the second distance. The first facial contour key point in the corresponding position includes the eyeball key point located in the left facial region of the face region, and the first facial contour key point corresponding to the eyeball key point is the eyeball at the first facial key point in the left facial region means a first facial contour key point located at the closest distance to the key point; If the eye key point is located in the right face region of the face region, the first facial contour key point corresponding to the eye key point is the first face contour key point located closest to the eye key point in the first facial contour key point of the right face region Means the outline key point. the midpoint of the key point of the nostril of the corresponding position is: if the key point of the eyeball is located in the left facial region of the face region, the midpoint of the key point of the nostril corresponding to the key point of the eye is the midpoint of the key point of the nose of the left face region; When the eyeball key point is located in the right face region of the face region, the midpoint of the nose key point corresponding to the eyeball key point means the midpoint located at the nostril key point in the right face region.

The calibration parameter is generated according to the ratio. When the ratio of the first distance and the second distance is 1, the midpoint of the first facial contour key point corresponding to the eyeball key point and the nose bridge key point corresponding to the eyeball key point is set as the first midpoint.

If the ratio between the first distance and the second distance is greater than or less than 1, the midpoint (to be identified below (referred to as a first midpoint) to a symmetrical centerline or may first determine an offset amount relative to the facial contour, said offset amount comprising a lateral offset amount and a vertical offset amount. Let the point be the first intermediate point determined by the sum of the offset amount and the coordinates of the first intermediate point to be identified.

In one possible implementation form, if the first distance is not equal to the second distance, the midpoint to be identified through the offset amount may be moved to the facial contour, for example an eye key point and a first facial contour key point is located in the left face region, the lateral offset amount takes a negative value, causing the first midpoint to be identified to be shifted to the face outline; if the eye key point and the first facial contour key point are located in the left facial region, the lateral offset amount takes a positive value, causing the first midpoint to be identified to be moved to the symmetrical center line; if the eye key point and the first facial contour key point are located in the right facial region, the lateral offset amount takes a positive value, causing the first midpoint to be identified to be moved to the facial contour; If the eye key point and the first facial contour key point are located in the right face region, the lateral offset amount takes a negative value, causing the first midpoint to be identified to be moved to the symmetrical center line.

If the target area is a circular area, the target area may be determined from the face area based on the first midpoint and the radius of the target area set by the user. In the implementation manner of determining the radius of one target region, the radius of the target region is determined according to the area of the face region. In another implementation manner of determining the radius of the target area, the radius of the target area may also be obtained by estimation by a deep learning model according to the first face image.

In addition, the step S120,

The method further includes determining the target area based on the first midpoint and the key point of the corner of the eye.

In one possible implementation form, a circular radius or major and minor axis of the target area is calculated by obtaining the coordinates of the first midpoint and the corner key point in the first face image using a linear interpolation algorithm. If the target region is circular, the target region can be configured by using the first intermediate point as the centrifugal point of the circular region and taking the calculated circular radius of the target region as the radius. If the target area is an ellipse, since the first midpoint is the center of the elliptical area, one of the major axis or the minor axis is determined, and then the other axis is also determined based on a preset ratio between the major axis and the minor axis, and includes a center, a major axis and a minor axis. When all of these are determined, the elliptical target area is also determined accordingly.

The above is only an example of a linear interpolation algorithm, and a specific implementation is not limited to any one of the above.

For the side face image, the step S120 is

If it is determined that the direction indicates that the first face image is a side face image, determining a second midpoint of the target area based on a second facial contour key point and a nose bridge key point.

When the apple zone adjustment is performed, since the face directions are different, a target area can be determined by selecting different key points.

For example, the second facial contour key point in the side face image is different from the first facial contour key point in the front face image. The second facial contour key point may be a facial contour key point whose ordinate is within a second target range, wherein the second target range is the ordinate of the area covered by the apple zone and the fifth preset range. sum, and the fifth preset range is smaller than the third preset range. Optionally, key point 4, key point 5, key point 27 and key point 28 shown in FIG. 4 are all second facial contour key points.

In this embodiment, the key point of the nostril is the same as the key point of the nose of the front face image, and can be referred to by selecting the key point 80 or the key point 81 shown in FIG. 4 .

In this embodiment, the second midpoint may likewise be the midpoint of the second facial contour key point and the nostril key point (referred to as the second midpoint to be identified below), or an offset amount in the frontal reference face image. The second midpoint is obtained by moving the second midpoint to be identified in the side face image in such a way that the first midpoint to be identified is moved using the method to obtain the first midpoint.

The second midpoint may be a center point of the target area.

In addition, the step S120 may further include determining the range of the target area based on the second midpoint and the key point of the corner of the eye.

In the present embodiment, the second midpoint and the key point of the corner of the eye may obtain a parameter of a determination range, such as a radius of the target area, based on a linear interpolation algorithm, thereby determining the range of the target area in the first face image. decide

In some embodiments, the step S130,

Based on the key point and the direction, it may include determining an adjustment parameter of the apple zone.

In order to implement the optimal adjustment of Apple Zone, specific adjustment parameters can be determined based on the key point and direction.

According to different directions, Apple Zone's adjustment strategy may be different. For example, in the adjustment of the front face image and the adjustment of the side face image, at least one of the adjustment direction and the angle is different, and the corresponding adjustment strategy is also different. Again, for example, if the directions are different, the adjustment of at least one of the brightness and the gradient of the Apple Zone is different.

For example, the step S130 may specifically include determining an adjustment direction of the apple zone based on the key point and the direction.

If the directions are different, the direction of adjustment of the apple zone is different.

When image beautification is performed, it is generally necessary to perform pull-up or swell adjustment for the apple zone, and since the face direction is different, the specific direction of the pull-up or the specific direction of the swell is different.

In some embodiments, the determining of the adjustment direction of the apple zone based on the key point and the direction comprises:

determining that the adjustment direction of the Apple Zone is the first direction based on the key point when the direction indicates that the first face image is a front face image; and

when the direction indicates that the first face image is a side face image, determining that the adjustment directions of the apple zone are a second direction and a third direction based on the key point, wherein the third direction is different from the second direction includes -.

In some embodiments, when the third direction is different from the second direction, the third direction includes, but is not limited to, being perpendicular to the second direction.

For example, for a side face image, the target area is moved in two directions, one is to pull up the target area toward the eyes, the other is to move the target area toward the edge of the face, optionally in a third direction is perpendicular to the second direction.

However, in order to optimize the adjustment effect of the apple zone, in this embodiment, the first direction may be a direction in which the first midpoint points to the corner of the eye key point; The second direction may be a direction in which the second intermediate point points to the eye corner key point; The third direction is a direction opposite to the nostril key point pointed to by the second facial contour key point (ie, a direction pointing from the nostril key point to the second facial contour key point).

Since both the facial contour key points and the eye tail key points are detected from the first facial image, the distribution itself reflects the facial features, and in this way, based on the facial contour key points and the eye tail key points, the adjusted first direction; It is possible to determine the second direction and the third direction, perform individual adjustments for substantially different faces, and implement the optimal adjustment of the Apple Zone.

Specifically, the step S130,

Based on the key point of the corner of the eye and the key point of the head of the eye, determining the maximum moving distance of the apple zone may include.

the distance between the corner key point and the eyebrow key point is substantially the length of the eye in the first facial image; For example, if the distance between the eye corner key point and the eye head key point is M pixels, the maximum distance may be M*A pixels. A may be a positive number less than 1. Also, for example, the value of A may be a positive number between 0.44 and 0.74.

In some embodiments, the actual movement distance of the apple zone may be determined based on a user operation, for example, the user inputs the user operation into the human-machine interaction interface, and determines the actual movement distance according to the user operation. If the movement distance calculated based on the user manipulation is greater than the maximum movement distance, the Apple Zone is adjusted using the maximum movement distance as the actual movement distance of the Apple Zone.

In some embodiments, the step S130,

If the direction indicates that the first face image is a side face image, determining an actual moving distance of the apple zone of the side face image according to the face deflection angle and the maximum moving distance.

In this embodiment, the maximum movement distance of the Apple Zone may be determined based on the key point, and the maximum movement distance is the maximum movement distance of the target area.

In this embodiment, using the key point of the eye as the key point, the maximum moving distance of the apple zone is determined based on the size of the eye.

In one possible implementation form, a positive correlation between the actual moving distance and a face deflection angle in the side face image is shown. For example, in some embodiments, when the face deflection angle is 90 degrees or -90 degrees, the actual moving distance may be the maximum moving distance. When the face deflection angle is not 90 degrees and -90 degrees, the actual movement distance = the maximum movement distance * a, where a is a positive number less than 1.

Of course, the above is merely an example, and there are many specific implementations that are not limited thereto.

In some embodiments, the image processing method further comprises the following steps.

In step S150, a brightening process is performed on the apple zone in the second face image to obtain a third face image.

The second facial image causes Applezone to move in the face relative to the first facial image. In this embodiment, a brightening process is performed on the moved Apple Zone to make the Apple Zone appear fuller and fuller.

In some embodiments, in the brightening process, the brightening level of the pixel for different locations in the apple zone is different. For example, the brightness level from the highest point of the Apple Zone to the periphery of the Apple Zone is sequentially decreased, so that the location of the highest point of the Apple Zone has the highest brightness, and then the brightness is set to the same brightness as the skin pixels around the Apple Zone. It decreases gradually and slowly, on the one hand, making the Apple Zone look more natural after being adjusted, and at the same time using the difference in brightness at different positions of the Apple Zone to show the three-dimensional effect of the Apple Zone.

In some embodiments, the image processing method comprises:

obtaining a skin color parameter of the face from the second face image; and

based on the facial skin color parameter, further comprising determining a brightening parameter for the apple zone brightening treatment;

The step S150 may include performing a brightening process on the apple zone in the second face image based on the brightening parameter to obtain the third face image.

In one way, by performing brightening using a single brightening parameter for different face images, the original dark face image is very unnatural in the apple zone after brightening, but the original dark face image is very unnatural to the face image with light skin color. In contrast, the brightening effect of Apple Zone after brightening is unclear. In this embodiment, a skin color parameter of the face image is obtained, and the skin color parameter is

Brightness values of skin pixels in a face image and liveness values of skin pixels between eyes and lips in a face image, grayscale histogram of skin pixels in a face image, and vivid values of skin pixels between eyes and lips in a face image, in a face image contains at least one of the raw values of skin pixels and a grayscale histogram of skin pixels between eyes and lips in a face image, a grayscale histogram of skin pixels in a face image, and a grayscale histogram of skin pixels between eyes and lips in a face image However, the present invention is not limited thereto.

Based on the skin color parameter, it is possible to determine the average brightness or intermediate brightness of the skin around the Apple Zone and the Apple Zone. It is possible to differentiate the skin around the zone and at the same time reduce the difference between the skin around the apple zone and the apple zone.

The brightening parameter includes, but is not limited to, at least one of a brightening width and a brightening ratio.

There are a plurality of specific implementation methods of the brightening apple zone, and in this embodiment, brightening is performed using a mask image.

In some embodiments, brightening the Apple Zone comprises:

If one corresponding brightening width is added to the original pixel value of the apple zone based on the brightening width, the pixel value after brightening can be obtained. acquiring a zone, and acquiring a third face image;

In some other embodiments, brightening the Apple Zone comprises:

Based on the brightening ratio, the pixel value after brightening can be obtained by multiplying the original pixel value of the apple zone by the brightening ratio. , obtaining a third face image.

In some embodiments, the step S150 may include: generating a mask image according to the location of the apple zone in the second face image; and obtaining the third face image by performing the Apple Zone brightening process on the second face image based on the mask image.

The step S150 is,

determining the location of the apple zone in the second face image;

forming one mask image having the same dimensions as the second facial image based on the position, the mask image forming a translucent brightening halo at the position of the second facial image; Other locations are transparent areas - ; and

The method may include generating the third face image by superimposing the mask image on the second face image.

In this case, since the brightness value of the brightening halo in the mask image is the brightness value of the apple zone immediately after brightening, the brightening parameter may further include the brightness value of the apple zone after brightening.

7 is a schematic diagram of a mask image provided in this embodiment. In FIG. 5 , the white halo area overlaps the area where the apple zone adjusted to brighten the apple zone is located.

Another method of performing brightening for the Apple Zone includes the steps of generating a mask image according to the location of the Apple Zone in the second face image,

according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image, generating the mask image including the bright point;

and,

obtaining the third face image by performing the Apple Zone brightening process on the second face image based on the mask image;

obtaining a mixed second pixel value by mixing pixels having a pixel value greater than a predetermined threshold value of the mask image with a first pixel value of a position pixel corresponding to the second face image; and

based on the second pixel value, generating the third face image.

In some embodiments, the bright point of the apple zone may be predetermined, and both the shape and brightness of the bright dot may be predetermined. In this way, the speed of generating the mask image can be accelerated.

The position of the bright dot of the apple zone in the mask image corresponds to the position of the apple zone in the second face image. For example, the detection of key points in the first face image is performed, and the first face image and the second face image are partially transformed only in the apple zone, but the transformation is still within the face range, and there is no large movement, so 2 The area where the apple zone is located in the face image may be determined according to the key point detected in step S110; For example, according to the target area determined in step S120; Alternatively, an area in which the apple zone is located in the second face image is obtained based on the target area and the actual moving distance, and the corresponding bright point is added to the mask image based on the area.

In the present embodiment, the pixel value at the position of the bright point in the mask image is greater than the predetermined threshold, and all pixel values other than the pixel value at the position of the bright dot are less than or equal to the predetermined threshold. When performing pixel blending in this way, it is possible to use a computation module such as a graphics processing unit (GPU) suitable for a large amount of computation, so that when traversing the pixels in the mask image, the pixel value is compared with a predetermined threshold. , determine whether the pixel should be blended with the pixel in the second face image. For example, if the second face image is an RGB image, the second face image contains three color channels, each pixel of the three color channels all have a corresponding pixel value, and the Mth pixel in the mask image If the pixel value is greater than the predetermined threshold value, after performing blending on the pixel value of the M-th pixel in the mask image and the production values of the three color channels of the M-th pixel in the second face image, these three colors The second pixel value is obtained by combining the mixed values of the channels. Here, the mixing may be a linear mixing or a non-linear mixing. In this embodiment, the mixing of the mask image and the second face image is preferably non-linear mixing, and the brightening effect of the apple zone obtained in this way is better.

A specific functional relationship of the nonlinear mixing may be as follows.

Sqrt(original color)*(a*mask color-1.0)+b*original color*(c-mask color), where original color is the first pixel value in the second face image; mask color is the pixel value in the mask image; a, b and c are all known computational parameters. The specific value can be set according to the request.

Sqrt means square root solution, and Sqrt (original color) means square root solution for the first pixel value.

In this embodiment, the value of a and the value of b may be the same, for example, after both the original color and the mask color are normalized to a value between 0 and 1, the values of a and b are both 2.0 can be; The value of c may be 1.0.

In other words, the values of a, b, and c may be the same or different.

In some embodiments, normalization processing is performed on pixel values or product values before mixing the pixel values, so that the pixel values or product values obtained are all data located between 0 and 1. In this case, on the one hand, even if an image is formed by color channels of different bit numbers, the same pixel value or product value located between 0 and 1 may be obtained. On the other hand, through the normalization process, the value involved in the subsequent pixel blending calculation is greatly reduced, thereby simplifying the calculation.

In some embodiments, the predetermined threshold may be a value such as 0.4, 0.45, 0.5, 0.55, and 0.6. The range of the predetermined threshold value may be 0.4 to 0.6.

Also, the image processing method further comprises: obtaining a control parameter;

Based on the second pixel value, generating the third face image includes:

performing linear mixing based on the control parameter, the second pixel value, and the first pixel value to obtain a third pixel value; and

and generating the third face image based on the third pixel value.

In this embodiment, the control parameter may be an external control parameter received from a human-machine interaction interface or another device by the imaging device, and may be used to control the brightening level of the control apple zone, and the level control parameter of the apple zone brightening. am.

After obtaining the second pixel value, linear mixing is performed based on the control parameter and the first pixel value to obtain the third pixel value. For example, if the control parameter is a ratio parameter, it may be based on A(RGB1)+RGB2=RGB3. For example, A may be the control parameter; the RGB1 may be the original first pixel value of the second face image; RGB2 may be the mixed second pixel value; RGB3 may be a third pixel value that forms the third face image after mixing again.

In this embodiment, the third face image is not generated using the second pixel value directly, but is linearly mixed again using the original second face image and the mixed second pixel value to obtain the third face image, , In this way, the brightening of Apple Zone can be more natural, and the image processing effect is better.

It should be understood that the embodiment of the present invention describes in detail the adjustment of the target region of the apple zone in the face image, and in actual application, based on the technical solution provided in the embodiment of the present invention, the forehead region, the chin region, It can be adjusted for the cheekbone area, etc.

As shown in Fig. 8, the image processing apparatus 1 provided by the present embodiment includes:

a detecting unit 11 for detecting a key point of the first face image;

a first determining unit (12) for determining, based on the key point, a target area adjusted by the Apple Zone;

a second determining unit (13) for determining, based on the key point, an adjustment parameter of the apple zone; and

and an adjustment unit (14) for adjusting the target area to form a second face image, based on the adjustment parameter.

Combined with any embodiment of the present invention, the image processing device 1 comprises:

a third determining unit (15) for determining a direction of a face in the first face image;

The first determining unit 12,

based on the key point and the direction, to determine the target area adjusted by the applezone.

Combined with any of the embodiments of the present invention, the key points include an eye corner key point, a first facial contour key point and a nostril key point;

The second determining unit 13,

if it is determined that the direction indicates that the first facial image is a frontal facial image, determine a first midpoint of the target area based on the first facial contour key point and the nostril key point;

Based on the first midpoint and the key point of the corner of the eye, the target area is determined.

In combination with any of the embodiments of the present invention, the key points include an eye corner key point, a second facial contour key point and a nostril key point;

The first determining unit 12,

if it is determined that the direction indicates that the first face image is a side face image, determine a second midpoint of the target area based on the second facial contour key point and the nostril key point;

Based on the second midpoint and the key point of the corner of the eye, the target area is determined.

Combined with any embodiment of the present invention, the second determining unit 13 comprises:

Based on the key point and the direction, to determine the adjustment direction of the apple zone.

Combined with any embodiment of the present invention, the third determining unit 15 comprises:

When the direction indicates that the first face image is a front face image, it is determined that the adjustment direction of the Apple Zone is the first direction based on the key point.

Combined with any embodiment of the present invention, the third determining unit 15 also

when the direction indicates that the first face image is a side face image, determining that the adjustment directions of the apple zone are a second direction and a third direction based on the key point, wherein the third direction is the second direction different from the direction

Combined with any embodiment of the present invention, the first direction is a direction in which the first midpoint of the target area points to the corner key point.

Combined with any of the embodiments of the present invention, the second direction is a direction in which the second midpoint of the target area points to the corner key point.

The third direction is a direction in which the nose bridge key point points to the second facial contour key point determining the target area.

Combined with any embodiment of the present invention, the second determining unit 13 comprises:

based on the key point and the direction, to determine an adjustment width of the apple zone.

Combined with any embodiment of the present invention, the second determining unit 13 comprises:

determining the maximum moving distance of the apple zone based on the key points of the corners of the eyes and the key points of the corners of the eyes;

When the direction indicates that the first face image is a side face image, an actual moving distance of the apple zone of the first face image is determined according to a face deflection angle and the maximum moving distance.

Combined with any embodiment of the present invention, the image processing device 1 comprises:

and a brightening processing unit 16 for performing brightening processing on the apple zone in the second facial image to obtain a third facial image.

Combined with any embodiment of the present invention, the brightening processing unit 16 comprises:

obtain a skin color parameter of the face from the second face image;

determine a brightening parameter for apple zone brightening treatment based on the facial skin color parameter;

Brightening processing is performed on the apple zone in the second face image based on the brightening parameter to obtain the third face image.

Combined with any embodiment of the present invention, the brightening processing unit 16 comprises:

generate a mask image according to the position of the apple zone in the second face image;

Based on the mask image, the Apple Zone brightening process is performed on the second face image to obtain the third face image.

Combined with any embodiment of the present invention, the brightening processing unit 16 comprises:

This is to generate the mask image including the bright point according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image.

Combined with any embodiment of the present invention, the brightening processing unit 16 comprises:

a pixel having a pixel value greater than a predetermined threshold value of the mask image is mixed with a first pixel value of a position pixel corresponding to the second face image to obtain a mixed second pixel value;

Based on the second pixel value, the third face image is generated.

Combined with any embodiment of the present invention, the image processing device 1 comprises:

an acquiring unit 17 for acquiring the control parameter;

The brightening processing unit 16,

perform linear mixing based on the control parameter, the second pixel value, and the first pixel value to obtain a third pixel value;

Based on the third pixel value, the third face image is generated.

Next, two specific examples are provided based on the above embodiment.

In Example 1,

The apple zone adjustment method may include the following steps.

1) Pull up the Apple Zone area in the first face image.

2) In addition to the pull-up when the first face image is a side face image, also extend this area outward according to the face deflection angle of the first face image.

3) Use a mask texture to brighten the area where the Apple Zone is located.

As shown in FIG. 9 , a second face image is obtained by performing an apple zone pull-up or lateral expansion operation on an area in which the apple zone of the first face image is located. In FIG. 9 , the dotted circular area of the second face image with respect to the first face image may be the target area in which the apple zone is located; After merging the second mask image and the second face image, a third face image in which the Apple Zone is brightened is obtained.

In Example 2,

Taking the left face region in the first face image as an example, an example adjusted by Applezone is provided, and includes the following.

1. When the first face image is a front face image, an area within the apple zone (corresponding to the target area) is drawn from the first face image.

1) Calculate the midpoint of key point 5 and key point 80 as the midpoint of the pooling area (circular area), the maximum travel distance of the pooling area is set to k times the distance between key point 52 and key point 55, where k is positive is an integer, and configuration can be performed based on actual needs;

2) the pull-up direction is from the midpoint of the pulling area to the key point 52 (target point), and the range (radius) of the pulling area is obtained by linear interpolation from the midpoint to the target point;

2. When the first face image is a side face image, in addition to pulling up the area where the apple zone is located, this area is also extended outward according to the face deflection angle of the first face image.

1) One extension area can be determined based on the face deflection angle of the first face image, and the key point 4 is the midpoint of the extension area, and the maximum moving distance of the extension area is equal to the pulling distance, and at the same time the face deflection angle and the There is a positive correlation.

2) Let key point 4 be the midpoint of the extension region, the extension direction is opposite to the direction from key point 4 to key point 80, and the extent (radius) of the extension region is obtained by linear interpolation from the midpoint to the target point do.

3. Use the mask texture to brighten the area of the apple zone.

1) Finally perform a brightening process, that is, using a single mask texture, the pattern emphasizes the standard face apple zone area, and then blends the color with the first face image to brighten the apple zone area of the face, make it look more three-dimensional.

As shown in FIG. 10 , the imaging device provided by this embodiment includes:

Memory; and

a processor, wherein the processor is coupled to the memory and executes computer-executable instructions located in the memory to implement the image processing method provided in any of the foregoing technical solutions, for example, FIG. 1 and This is to implement the image processing method shown in at least one of FIG. 3 .

The memory may be various types of memory, such as RAM, ROM, and flash memory. The memory is for storing information, for example, computer-executable instructions and the like. The computer-executable instructions may be various program instructions, for example, at least one of a target program instruction and a source program instruction.

The processor may be any type of processor, such as a central processor, microprocessor, digital signal processor, programmable array, digital signal processor, application specific integrated circuit, or image processor.

The processor is coupled to the memory via a bus. The bus may be an integrated circuit bus or the like.

In some embodiments, the imaging device may further include a communication interface, and the communication interface may include a network interface, for example, a LAN interface, a transmission/reception antenna, and the like. Likewise, the communication interface may be connected to the processor and used to transmit and receive information.

In some embodiments, the imaging device further includes a human-machine interaction interface, for example, the human-machine interaction interface may include various input and output devices, for example, a keyboard and a touch screen. .

An embodiment of the present invention provides a computer storage medium having computer executable code stored thereon; After the computer-executable instructions are executed, an image processing method such as the image processing method shown in at least one of FIGS. 1 and 2 provided by one or a plurality of technical methods in an image device, a database, and a first private network can be applied to

The computer storage medium may include various media having a recording function, such as a CD, a floppy disk, a hard disk, a magnetic tape, an optical disk, a U disk, or a mobile hard disk various storage media. Optionally, the computer storage medium may be a non-transitory storage medium, and the computer storage medium is readable by a processor, so that the computer-executable instructions stored in the computer storage medium are obtained and executed by the first processor, An image processing method provided in any technical solution is implemented, for example, an image processing method applied to an image device or an image processing method applied to an application server is executed.

This embodiment further provides a computer program product, wherein the computer program product includes computer-executable instructions; After the computer-executable instruction is executed, the image processing method provided in one or more of the above-described technical solutions may be implemented, including, for example, at least one image processing method illustrated in FIGS. 3 and 5 .

a computer program tangibly contained in the computer storage medium, wherein the computer program includes program code for executing the method shown in the flowchart, the program code comprising instructions corresponding to the method steps provided in the embodiments of the present invention. may include

In the various embodiments provided herein, it is to be understood that the disclosed apparatus and method may be implemented in other ways. The device embodiment described above is merely exemplary, for example, the division of the unit is only logical function division, and there may be other division methods in actual application, for example, a plurality of units or components are It may be combined or integrated into the system, or some features may be omitted or not implemented. In addition, the mutual coupling or direct coupling or communication connection of each component part shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

A unit described as a separate part may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, or may be in one location or distributed in a plurality of network units; ; According to actual needs, some or all of the units may be selected to implement the purpose of the scheme of the present embodiment.

In addition, each functional module in each embodiment of the present invention may all be integrated into one processing unit, each unit may be used alone as one unit, and two or more units may be integrated into one unit. can; The integrated unit may be implemented in the form of hardware or may be implemented in the form of a software functional unit.

A person skilled in the art will know that all or part of the operations for implementing the embodiment of the method may be completed through hardware related to program instructions, and the above-described program may be stored in a computer-readable storage medium, and when the program is performed, performing includes operation of the method embodiment; The aforementioned storage medium includes various media capable of storing a program code such as a mobile storage device, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The above description is only specific embodiments of the present invention, and the protection scope of the present invention is not limited thereto. Those of ordinary skill in the art to which the present invention pertains will readily appreciate that all changes or replacements within the technical scope disclosed in the present invention must fall within the protection scope of the present application. Accordingly, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (36)

An image processing method comprising:
detecting a key point of the first face image;
determining a target area adjusted by the Apple Zone based on the key point;
determining an adjustment parameter of the Apple Zone based on the key point;
adjusting the target area based on the adjustment parameter to form a second face image;
generating a mask image including the bright point according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image; and
and obtaining a third face image by performing the Apple Zone brightening process on the second face image based on the mask image. According to claim 1,
Before determining the target area adjusted by the Applezone based on the key point, the image processing method includes:
determining the orientation of the face in the first face image;
Determining the target area adjusted by the Apple Zone based on the key point comprises:
and determining the target area adjusted by the applezone based on the key point and the direction. 3. The method of claim 2,
the key points include an eyebrow key point, a first facial contour key point, and a nostril key point;
Determining the target area adjusted by the Apple Zone based on the key point and the direction comprises:
when it is determined that the direction indicates that the first facial image is a frontal facial image, determining a first midpoint of the target area based on the first facial contour key point and the nostril key point; and
and determining the target area based on the first midpoint and the key point of the corner of the eye. 3. The method of claim 2,
the key points include an eyebrow key point, a second facial contour key point, and a nostril key point;
Determining the target area adjusted by the Apple Zone based on the key point and the direction comprises:
when it is determined that the direction indicates that the first face image is a side face image, determining a second midpoint of the target area based on the second facial contour key point and the nostril key point; and
and determining the target area based on the second midpoint and the key point of the corner of the eye. 3. The method of claim 2,
Determining the adjustment parameter of the Apple Zone based on the key point comprises:
and determining an adjustment direction of the apple zone based on the key point and the direction. 6. The method of claim 5,
Determining the adjustment direction of the apple zone based on the key point and the direction comprises:
and determining that the adjustment direction of the Apple Zone is the first direction based on the key point when the direction indicates that the first face image is a front face image. 6. The method of claim 5,
Determining the adjustment direction of the apple zone based on the key point and the direction comprises:
when the direction indicates that the first face image is a side face image, determining that the adjustment directions of the apple zone are a second direction and a third direction based on the key point - the third direction is different from the second direction An image processing method comprising: 7. The method of claim 6,
The first direction is an image processing method, characterized in that the first intermediate point of the target area is a direction in which the key point of the corner of the eye points. 8. The method of claim 7,
the second direction is a direction in which a second midpoint of the target area points to an eye corner key point; The third direction is an image processing method, characterized in that the nose bridge key point points to the second facial contour key point for determining the target area. 3. The method of claim 2,
Determining the adjustment parameter of the Apple Zone based on the key point comprises:
and determining an adjustment width of the apple zone based on the key point and the direction. 11. The method of claim 10,
Determining the adjustment width of the apple zone based on the key point and the direction comprises:
determining the maximum moving distance of the apple zone based on the key point of the corner of the eye and the key point of the corner of the eye; and
when the direction indicates that the first face image is a side face image, determining an actual moving distance of the apple zone of the first face image according to a face deflection angle and the maximum moving distance Image processing method. delete delete delete delete According to claim 1,
Acquiring the third face image by performing the apple zone brightening process on the second face image based on the mask image,
obtaining a mixed second pixel value by mixing a pixel of the mask image whose pixel value is greater than a predetermined threshold with a first pixel value of a position pixel corresponding to the second face image; and
and generating the third face image based on the second pixel value. 17. The method of claim 16,
The image processing method is
further comprising obtaining a control parameter;
Based on the second pixel value, generating the third face image includes:
performing linear mixing based on the control parameter, the second pixel value, and the first pixel value to obtain a third pixel value; and
and generating the third face image based on the third pixel value. An image processing device comprising:
a detection unit for detecting a key point of the first face image;
a first determining unit for determining, based on the key point, a target area adjusted by the apple zone;
a second determining unit for determining, based on the key point, an adjustment parameter of the apple zone;
an adjustment unit for adjusting the target area to form a second face image based on the adjustment parameter; and
generating a mask image including the bright point according to the preset bright point of the apple zone brightening and the position of the apple zone in the second face image; and a brightening processing unit configured to perform the apple zone brightening process on the second face image to obtain a third face image based on the mask image. An imaging device comprising:
Memory; and
18. A processor, coupled to said memory, for executing computer-executable instructions stored in said memory to implement the method according to any one of claims 1 to 11, 16 or 17. imaging device. A computer storage medium storing computer-executable instructions, comprising:
A computer storage medium, characterized in that after the computer-executable instructions are executed, the method according to any one of claims 1 to 11, 16 and 17 can be implemented. As a computer program stored in a computer-readable storage medium,
A computer program, characterized in that it implements the method according to any one of claims 1 to 11, 16 and 17 when the computer program is executed by a processor. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images