WO2022042160A1

WO2022042160A1 - Image processing method and apparatus

Info

Publication number: WO2022042160A1
Application number: PCT/CN2021/108227
Authority: WO
Inventors: 毕星; 汪洋; 杨浩; 王剑
Original assignee: 北京达佳互联信息技术有限公司
Priority date: 2020-08-24
Filing date: 2021-07-23
Publication date: 2022-03-03
Also published as: CN114095647A

Abstract

Embodiments of the present application provide an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining a target image comprising a target part to be beautified; obtaining coordinate values, in the target image, of contour feature points of the target part to be beautified as target coordinate values; according to the target coordinate values and a predetermined configuration file, obtaining a grid structure of the target part to be beautified as a target grid structure; and according to the target grid structure, beautifying the target part to be beautified to obtain a beautified image.

Description

Image processing method and device

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure claims the priority of the Chinese patent application with the application number "202010859989.7" filed on August 24, 2020.

technical field

The present disclosure relates to the field of computer technology, and in particular, to an image processing method and apparatus.

Background technique

With the development of smart phones, the performance of mobile phone computing and photography has been greatly improved. Especially in recent years, the more advanced AI processing functions have enabled mobile terminals to perform real-time operations, resulting in the birth of a large number of camera and video recording applications with special effects functions, which can realize various special effects functions, such as beauty special effects and so on.

In the related art, in order to achieve the beauty effect, the center point of the part to be beauty makeup can be determined first, and then the center point of the part to be beauty makeup is positioned, and then the beauty treatment is performed. If the structure of the part to be beautified is complex, the found center point of the part to be beautified may be inaccurate, so that the cosmetic effect may be poor.

Therefore, how to improve the cosmetic effect of the part to be beautified has become an urgent problem to be solved at present.

SUMMARY OF THE INVENTION

The present disclosure provides an image processing method, apparatus, electronic device and storage medium.

In some embodiments, an image processing method is provided, comprising:

Obtain the target image containing the target cosmetic part;

Obtain the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

According to the target coordinate value and the predetermined configuration file, the grid structure of the target cosmetic part is obtained as the target grid structure;

According to the target grid structure, beauty makeup processing is performed on the beauty makeup part in the target image to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be beautified and coordinate values of vertices of the preset triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The vertex of the triangle includes two contour feature points of the part to be beautified and a first expansion point, and the first expansion point is a point set in advance according to the contour of the part to be beautified.

In some embodiments, according to the target coordinate value and a predetermined configuration file, the grid structure of the target cosmetic part is obtained, including:

According to the target coordinate value and the coordinate value of the vertex of the first type of triangle, the coordinate value of the first type of extension point is calculated, wherein the first type of extension point is in the target mesh structure, and the point corresponding to the first extension point;

According to the target coordinate value and the coordinate value of the first type of extension point, the target grid structure is obtained.

In some embodiments, the target mesh structure includes at least one first target triangle, the first target triangle is in one-to-one correspondence with the first type of triangle, and the first type of triangle corresponds to the corresponding The first target triangles are similar; the first target triangles are in one-to-one correspondence with the first type extension points, and the first type extension points belong to the vertices of the corresponding first target triangles;

According to the target coordinate value and the coordinate value of the vertex of the first-type triangle, the coordinate value of the first-type extension point is calculated and obtained, including:

According to the similarity relationship between the first type of triangle and its corresponding first target triangle, as well as the target coordinate value and the coordinate value of the vertex of the first type of triangle, the coordinates of the first type of extension point are calculated and obtained value.

In some embodiments, obtaining the target grid structure according to the target coordinate value and the coordinate value of the first type of extension point includes:

According to the coordinate value of the first type of extension point and the target coordinate value, the coordinate value of the second type of extension point is calculated, wherein the second type of extension point is the target grid structure divided by the target Contour feature points of the part to be beautified and points other than the first type of extension points;

The target grid structure is obtained according to the target coordinate value, the coordinate value of the first type extension point and the coordinate value of the second type extension point.

In some embodiments, the preset triangle further includes at least one triangle of the second type, wherein a vertex of a triangle of the second type includes a second extension point and two points in the target set, wherein the target set Including the contour feature point of the part to be beautified and the first expansion point, the second expansion point is a point set in advance according to the contour of the part to be beautified;

The target mesh structure also includes at least one second target triangle, the second target triangle is in one-to-one correspondence with the second type of triangle, and the second type of triangle is similar to its corresponding second target triangle , wherein the second target triangle is in one-to-one correspondence with the second type of extension point, and the second type of extension point belongs to the vertex of the corresponding second target triangle;

According to the coordinate value of the first type of expansion point and the target coordinate value, the calculation to obtain the coordinate value of the second type of expansion point, including:

According to the similarity relationship between the second type of triangle and its corresponding second target triangle, and the target coordinate value, the coordinate value of the first type of extension point, the coordinate value of the vertex of the first type of triangle and The coordinate value of the vertex of the second type of triangle is calculated to obtain the coordinate value of the second type of extension point.

In some embodiments, calculating the coordinate value of the second type of extension point according to the coordinate value of the first type of extension point and the target coordinate value, including:

Using a smooth transition or interpolation method, the coordinate value of the second type of expansion point is calculated according to the coordinate value of the first type of expansion point and the target coordinate value.

In some embodiments, an image processing apparatus is provided, comprising:

an image acquisition module, configured to acquire a target image including the target cosmetic part;

A coordinate value acquisition module, configured to acquire the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

a grid structure acquisition module, configured to obtain the grid structure of the target cosmetic part according to the target coordinate value and a predetermined configuration file, as the target grid structure;

a drawing module, configured to perform beauty makeup processing on the beautified part in the target image according to the target grid structure to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be makeup and coordinate values of vertices of the preset target triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The triangle-like vertex includes two contour feature points of the part to be beautified and a first extension point, and the first extension point is a point set in advance according to the contour of the part to be beautified.

In some embodiments, the grid structure acquisition module includes:

The first extension point calculation sub-module is configured to calculate the coordinate value of the first type of extension point according to the target coordinate value and the coordinate value of the vertex of the first type of triangle, wherein the first type of extension point The point is the point corresponding to the first expansion point in the target grid structure;

The grid structure drawing sub-module is configured to obtain the target grid structure according to the target coordinate value and the coordinate value of the first-type extension point.

The first extension point calculation submodule is specifically configured as:

In some embodiments, the grid structure drawing submodule includes:

The second extension point calculation unit is configured to calculate the coordinate value of the second type of extension point according to the coordinate value of the first type of extension point and the target coordinate value, wherein the second type of extension point is the Other points in the target grid structure except the contour feature points of the target beautified part and the first type of extension points;

The grid structure drawing unit is configured to obtain the target grid structure according to the target coordinate value, the coordinate value of the first type of extension point and the coordinate value of the second type of extension point.

The second extension point calculation unit is specifically configured as:

In some embodiments, the second extension point calculation submodule is specifically configured as:

In some embodiments, an electronic device is provided, comprising:

processor;

a memory for storing the processor-executable instructions;

Wherein, the processor is configured to execute the instructions to implement the following steps:

Obtain the target image containing the target cosmetic part;

In some embodiments, a storage medium is provided, and when the instructions in the storage medium are executed by a processor of an electronic device, the following steps are implemented:

Obtain the target image containing the target cosmetic part;

In some embodiments, there is provided a computer program product comprising instructions for implementing the following steps when the computer program product is run on a computer:

Obtain the target image containing the target cosmetic part;

In the embodiment of the present disclosure, two contour feature points of the part to be beautified and a first extension point are used to form the first type of triangles, and they are used as the preset triangles in the mesh structure of the part to be beautified, so that these pre-made triangles are Set the coordinate value of the triangle to form a configuration file, and then use the configuration file and the coordinate value of the contour feature point of the target to be cosmetic part in the target image to calculate the target grid structure of the target to be cosmetic part, and then according to The target grid structure performs beauty processing on the target beauty makeup part to obtain a beauty makeup image.

It can be seen from this that in the embodiment of the present disclosure, the first extension points are manually set on the basis of the contour feature points of the part to be beautified, and the grid of the part to be beautified is constructed from these contour feature points and the first extension point. Triangles in structure. That is, in the embodiment of the present disclosure, based on the contour feature points of the cosmetic part, the grid structure of the cosmetic part is constructed in advance, that is, a grid structure matching the cosmetic part is constructed, that is, a configuration file is constructed, so that when the need arises, the grid structure is constructed. When processing the target image including the target cosmetic part, the triangles in the configuration file can be directly used to generate a mesh structure matching the target cosmetic part. There is no need to locate the center point of the part to be beautified, so even if the AI point of the part to be beautified is not radiated, the grid structure matching the part to be beautified can still be obtained according to the pre-determined configuration file. To a certain extent, it can improve the beauty effect of the target beautified part.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

Fig. 1 is a schematic diagram of a beauty makeup process in the prior art;

Fig. 2 is the process schematic diagram of creating extension point in the prior art;

3 is a schematic flowchart of generating a grid structure based on an original image in the prior art;

Figure 4 is a schematic diagram of a texture material for eyelash effect;

Fig. 5 is the Mesh structure corresponding to the texture material shown in Fig. 4;

FIG. 6 is a schematic diagram of the mapping between the texture material shown in FIG. 4 and the Mesh structure shown in FIG. 5;

FIG. 7 is a schematic diagram of the effect of drawing the texture material shown in FIG. 4 into the original camera image;

8 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure;

9 is a schematic diagram of a first type of triangle included in a configuration file in an embodiment of the present disclosure;

10 is a schematic diagram of the verification result of a first expansion point in the present disclosure when the eyes are in different states;

11 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure;

Fig. 12 is the schematic diagram after adding the second expansion point in the first type triangle shown in Fig. 9;

13 is a schematic flowchart of a specific implementation manner of an image processing method provided by an embodiment of the present disclosure;

14 is a schematic diagram of an AI recognition point of a face identified in the present disclosure;

Figure 15 is a schematic diagram of a grid structure constructed based on the AI recognition points shown in Figure 14;

16 is a schematic diagram of the principle of checking an expansion point in the present disclosure;

17 is a structural block diagram of an image processing apparatus provided by the present disclosure;

18 is a structural block diagram of an electronic device provided by the present disclosure;

FIG. 19 is a structural block diagram of another electronic device provided by the present disclosure.

detailed description

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

As shown in Figure 1, the so-called beauty special effects, that is, the key points of the face are identified through the AI module, and then the pre-drawn beautification pictures are stretched according to the face structure, and then drawn to the original user's face. , in order to achieve the purpose of beautifying the appearance of the face. For example, draw eyelashes for users, modify lipstick color, etc. Among them, the stretching rule has a great influence on the final effect of beauty makeup. It affects the authenticity and accuracy of beauty makeup, and is the most critical part of the entire function.

Among them, regarding the construction of stretching rules, it is usually used to create extension points, which are then combined with AI points (ie, contour points of the part to be beautified) to form a complete set of detailed mesh structures (ie, Mesh structures).

In the prior art, the above-mentioned extension point generation method is described. As shown in Figure 2, a more conventional approach is to identify the AI point of the face for the original image; then, determine the center point of the face (or other specific organ); again, based on this center point, and then compare with the surrounding AI The identification points are connected, and based on this connection, the extension points are extended outward for a certain distance or proportion, and then the expansion points are obtained; finally, after the AI identification points and the expansion points are connected into several adjacent triangles, the Mesh structure is obtained. Among them, the center point can be calculated from multiple vertices. For example, the center point of the quadrilateral consisting of the top of the head, the chin, and the left and right ears can be used as the center point of the face, or the vertex representing the tip of the nose in the AI recognition points can be used as the center point of the face.

However, for complex organs, all extension points do not necessarily appear radial, and it is difficult to find the center point at this time. Although it can be expanded separately by selecting multiple center points, it may cause problems such as overlapping of Mesh meshes, and there are many disadvantages.

It can be seen that when the structure of the part to be beautified is complex, not all AI points are radial, which makes the center point of the part to be beautified inaccurate, resulting in the obtained mesh structure and the part to be beautified. mismatch, resulting in poor cosmetic results.

In order to make it easier to understand the image processing method and device of the embodiments of the present disclosure, the process of performing beauty makeup is first described as follows:

To complete the beauty function, two parts of input data are required: one is the raw image input from the camera. The second is a pre-made placement image and its corresponding static triangular mesh.

As shown in Figure 3, for the input image of the camera, the face in the image is first recognized by the AI module, and key points are marked in the key parts of the face. Such key points can be called AI recognition points; and then based on AI recognition A series of new extension key vertices (that is, extension points) are calculated through specific algorithm rules. In the case of having AI recognition points and extension points, they will be connected into several adjacent triangles, and these triangles will form a mesh structure.

For the placement image, it contains the corresponding Mesh structure, which is pre-configured. As shown in Figure 4, it is a texture material for eyelash effect. As shown in Figure 5, it is the Mesh structure corresponding to the texture material shown in Figure 4. Among them, as shown in Figure 6, the texture material will be drawn into the current camera input image according to the principle of one-to-one correspondence of triangles in the Mesh structure, that is, the image with the final effect shown in Figure 7 is obtained.

It can be understood that, the image processing method in the embodiment of the present disclosure can be executed by the image processing apparatus provided in the embodiment of the present disclosure, and the apparatus can be configured in an electronic device.

Referring to FIG. 8, an image processing method provided by an embodiment of the present disclosure is shown, and the image processing method may include the following steps:

Step 801: Acquire a target image including the target cosmetic part.

The target image may be a locally stored image, an image captured in real time by an electronic device, or one of the images in a locally stored video file, or one of the video files captured in real time by an electronic device. frame image.

Step 802: Acquire the coordinate values of the contour feature points of the target cosmetic part in the target image as the target coordinate values.

Wherein, the above-mentioned contour feature points are points located on the contour of the part to be beautified.

Therefore, in the embodiment of the present disclosure, the image processing apparatus can first determine the coordinate value of the contour feature point located in the target beautified part in the target image, and then can use the coordinate value as the target coordinate value.

Step 803 : According to the target coordinate value and the predetermined configuration file, obtain the grid structure of the target cosmetic part, which is used as the target grid structure.

It can be seen from this that, in the embodiment of the present disclosure, the configuration file includes multiple points, and the multiple points include contour feature points of the part to be beautified and manually set points (ie, extension points). For example, the first extension point in the configuration file for the eye may include points 91 to 910 as shown in FIG. 9 , and the contour feature points of the eye may include points 911 to 922 as shown in FIG. 9 . The multiple points included in the configuration file form multiple triangles according to certain rules. The rules for forming triangles are predetermined. For example, the points 91 to 922 shown in FIG. 9 follow the rule that two contour feature points and an extension point form a triangle to obtain the triangle shown in FIG. 9 . That is, the triangle shown in FIG. 9 is the first type of triangle.

Among them, the configuration file is used as the basis for generating the grid structure in the process of beautifying the part to be beautified. In some embodiments, using a known configuration file, in the case of beautifying the part to be beautified, the image processing device first needs to acquire an image of the part to be beautified, and then detects the beauty of the part to be beautified in the image. contour feature points, and then select the points corresponding to the contour feature points included in the configuration file from these contour feature points as input parameters, and then use the input parameters and configuration files to calculate the extension points that constitute the grid structure, and then use the input parameters and configuration files. All contour feature points included in the parameters and all calculated expansion points are drawn into a grid structure.

In addition, it should be noted that there is a one-to-one correspondence between the points included in the configuration file and the points in the target grid structure to be calculated, and the image processing device checks and calculates an extension point in the configuration file, which is Calculate the coordinates of the corresponding point in the target grid structure for this extension point in the configuration file.

In the embodiment of the present disclosure, the checking result of the first extension point is based on two contour feature points that belong to the same first type of triangle. The contour feature points belong to the points on the part to be beautified, so the check result of the first expansion point is more matched with the part to be beautified, so that the final target grid result is more matched to the part to be beautified.

In addition, as shown in Figure 10, points A, B, and C are three points in the configuration file of the eyes, and these three points form a first-type triangle, where points A and B are two contour feature points respectively, Point C is the first extension point, A1` and B1` are two contour feature points in the image with the eyes open and inclined, C1` is the check calculation of point C in the configuration file in the image with the eyes open and inclined Results; A2` and B2` are the two contour feature points in the image with the eyes closed and not tilted, C2` is the check result of point C in the configuration file in the image with the eyes closed and not tilted. It can be seen from this that, for a first extension point in the configuration file, under the different conditions of open eyes and tilted eyes, and closed eyes without tilted eyes, the verification results of the first extension point (that is, C1' and C1' in Fig. 10 and The coordinate value of C2`) is not much different. Therefore, in the embodiment of the present disclosure, a configuration file composed of a first-type triangle composed of two contour feature points and a first extension point can also be used to obtain a relatively stable grid under different states of the beautified part. structure.

Step 804: According to the target grid structure, perform beauty makeup processing on the beauty makeup part in the target image to obtain a beauty makeup image.

It can be seen from the above that, in the embodiment of the present disclosure, the image processing apparatus can use two contour feature points of the part to be beautified and a first extension point to form a first-type triangle, and use it as a triangle in the grid structure of the part to be beautified. Preset triangles, so that the coordinate values of these preset triangles form a configuration file, and then use the configuration file and the coordinate values of the contour feature points of the target cosmetic part in the target image to calculate and obtain the target cosmetic part. The target grid structure is obtained, and then beauty makeup processing is performed on the target beauty makeup part to obtain a beauty makeup image.

It can be seen from this that in the embodiment of the present disclosure, the first extension points are manually set on the basis of the contour feature points of the part to be beautified, and the grid of the part to be beautified is constructed from these contour feature points and the first extension point. Triangles in structure. That is, in the embodiment of the present disclosure, based on the contour feature points of the cosmetic part, the grid structure of the cosmetic part is constructed in advance, that is, a grid structure matching the cosmetic part is constructed, that is, a configuration file is constructed, so that when the need arises, the grid structure is constructed. In the case of processing the target image including the target cosmetic part, the triangles in the configuration file can be directly used to generate a mesh structure matching the target cosmetic part. There is no need to locate the center point of the part to be beautified, so even if the AI point of the part to be beautified is not radiated, the grid structure matching the part to be beautified can still be obtained according to the pre-determined configuration file. To a certain extent, it can improve the beauty effect of the target beautified part.

Referring to FIG. 11, an image processing method provided by an embodiment of the present disclosure is shown, and the image processing method may include the following steps:

Step 1101: Acquire a target image including the target cosmetic part.

Step 1102: Acquire the coordinate value of the contour feature point of the target cosmetic part in the target image as the target coordinate value.

Step 1103: Calculate the coordinate value of the first type of extension point according to the target coordinate value and the coordinate value of the vertex of the first type of triangle in the predetermined configuration file.

The first type of extension point is a point corresponding to the first extension point in the target grid structure, and the target grid structure is the grid structure of the target cosmetic part.

Step 1104: Obtain the target grid structure according to the target coordinate value and the coordinate value of the first type of extension point.

Wherein, in the embodiment of the present disclosure, a configuration file is pre-determined, and the configuration file includes a preset triangle in the mesh structure of the part to be beautified and the coordinate values of the vertices of the preset triangle, the preset triangle At least one triangle of the first type is included, and the vertex of a triangle of the first type includes two contour feature points of the part to be beautified and a first expansion point, and the first expansion point is pre-made according to the makeup. The point where the outline of the part is set.

Among them, the configuration file is used as the basis for generating the grid structure in the process of beautifying the part to be beautified. In some embodiments, using a known configuration file, in the case of beautifying the part to be beautified, the image processing device first needs to acquire the image of the part to be beautified, and then detects the part of the beautified part in the image. contour feature points, and then select the points corresponding to the contour feature points included in the configuration file from these contour feature points as input parameters, and then use the input parameters and configuration files to calculate the extension points that constitute the grid structure, and then use the input parameters and configuration files. All contour feature points included in the parameters and all calculated expansion points are drawn into a grid structure.

In addition, it should be noted that the points included in the configuration file have a one-to-one correspondence with the points in the target grid structure to be calculated, and then an extension point in the configuration file is checked and calculated, that is, the calculation configuration file The coordinates of the corresponding point in the target grid structure for this extension point in .

In the embodiment of the present disclosure, the verification result of the first extension point by the image processing apparatus is based on two contour feature points that belong to the same first type of triangle. The contour feature points belong to the points on the part to be beautified, so the check result of the first expansion point is more matched with the part to be beautified, so that the final target grid result is more matched to the part to be beautified.

In addition, as shown in Figure 10, points A, B, and C are three points in the configuration file of the eyes, and these three points form a first-type triangle, where points A and B are two contour feature points respectively, Point C is the first extension point. A1` and B1` are the two contour feature points in the open-eye and oblique image, C1` is the verification result of point C in the configuration file in the open-eye and oblique image; A2` and B2` are Two contour feature points in an image with eyes closed and not tilted, C2' is the result of checking the point C in the configuration file in an image with eyes closed and not tilted. It can be seen from this that, for a first extension point in the above configuration file, under different conditions of open eyes and tilted eyes, and closed eyes without tilted eyes, the verification results of the first extension point (that is, C1' and C1' in Fig. 10 and The coordinate value of C2`) is not much different. Therefore, in the embodiment of the present disclosure, a configuration file composed of a first-type triangle composed of two contour feature points and a first extension point can also be used to obtain a relatively stable grid under different states of the beautified part. structure.

Wherein, when two triangles are known to be similar, and the coordinate value of the vertex of one triangle and the coordinate value of the two vertices of the other triangle are known, the coordinate value of the unknown vertex of one triangle can be calculated.

In addition, in the embodiment of the present disclosure, the first type of triangle is similar to the first target triangle, and the vertices of the two are in one-to-one correspondence, that is, the two are similar and have a unique direction, so the calculated first expansion point is unique.

Step 1105: According to the target grid structure, perform beauty makeup processing on the beauty makeup part in the target image to obtain a beauty makeup image.

It can be seen from the above that, in the embodiment of the present disclosure, the image processing apparatus can manually set the first extension point on the basis of the contour feature points of the part to be beautified, and construct the beautified object from these contour feature points and the first extension point. Triangles in the mesh structure of the site. That is, in the embodiment of the present disclosure, based on the contour feature points of the cosmetic part, the grid structure of the cosmetic part is constructed in advance, that is, a grid structure matching the cosmetic part is constructed, that is, a configuration file is constructed, so that when the need arises, the grid structure is constructed. In the case of processing the target image including the target cosmetic part, the triangles in the configuration file can be directly used to generate a mesh structure matching the target cosmetic part. There is no need to locate the center point of the part to be beautified, so even if the AI point of the part to be beautified is not radiated, the grid structure matching the part to be beautified can still be obtained according to the pre-determined configuration file. To a certain extent, it can improve the beauty effect of the target beautified part.

According to the coordinate value of the first type of extension point and the target coordinate value, the coordinate value of the second type of extension point is calculated, wherein the second type of extension point is the target grid structure divided by the target The contour feature points of the part to be beautified and other points other than the first type of extension points;

It can be seen that, in the embodiment of the present disclosure, the image processing apparatus may continue to determine other extension points (ie, second extension points) after checking the first extension point in the configuration file, so that the obtained target grid The structure is closer to the shape and structure of the part to be beautified.

From this, it can be seen that the above configuration file may further include a second type of triangle formed by the contour feature point of the part to be beautified, two of the first extension points, and a second extension point. These second type of triangles are used to determine the second type of extension points in the target mesh structure described above.

Wherein, after the image processing device completes the checking and calculation of the first extension point in the configuration file and obtains the above-mentioned first type of extension point, in the case where it is known that the second type of triangle is similar to the second target triangle in the target mesh structure For example, for a second type triangle and a second target triangle, the coordinate value of the unknown point can be obtained according to the coordinate value of the known point in the two triangles, that is, the coordinate value of the second type extension point can be calculated.

For example, in the configuration file shown in FIG. 9 , a second extension point may be further added. For example, a schematic diagram of each point included in the configuration file after adding the second extension point may be as shown in FIG. 12 . Wherein, the point 923 and the point 924 shown in FIG. 12 are two second extension points, then the

points

924, 922, and 91 constitute a second-type triangle, and the

points

923, 917, and 95 constitute a second-type triangle.

In addition, since the points in the configuration file are in a one-to-one correspondence with the points in the target grid structure, for example, the grid structure formed by connecting the points in the configuration file shown in FIG. 12 can represent the open The grid structure of the image of the eye.

Wherein, after the image processing device completes the checking and calculation of the first extension point in the configuration file, that is, after obtaining the first type of extension point in the target grid structure, the image processing device can also obtain the first type of extension point in the target image according to the contour feature points of the part to be beautified in the target image, and The first type of extension point uses smooth transition or differential interpolation to extend other points, thereby reducing the amount of computation and improving the processing speed.

In addition, for the setting of extension points, the construction logic can be developed for a specific point through coding, the changes will be very flexible, and the performance of the performance is highly dependent on the code logic, which can achieve more peculiar and more natural effects. However, this method has a large workload, and the effect is flexible but the production is not flexible. However, this method can be mixed with the extension point generation scheme adopted in the embodiments of the present disclosure.

To sum up, the specific implementation manner of the image processing method provided by the embodiments of the present disclosure may be as follows:

As shown in Figure 13, a configuration file is provided in advance, the configuration file and AI recognition points are used as input data, and extended point verification is performed through the similar triangle algorithm, and finally a complete face (or other specific organs) beauty makeup details structure is obtained picture. This process needs to be calculated once for each frame of image. Among them, the AI recognition points of the face are shown in Figure 14, and the grid structure of the face is shown in Figure 15.

Wherein, the above-mentioned process of performing extended point verification by the similar triangle algorithm is as follows:

First of all, the image processing device can define how many points exist in the final grid structure to be constructed in the entire face beauty makeup (or specific organ beauty makeup), wherein this step is based on the requirements of the usage scene, and there is no fixed rule;

Secondly, the image processing apparatus can construct a configuration file, which contains the following three aspects of information: firstly, the relative position information of all points; secondly, a series of triangular structures composed of three points; thirdly, the The generation rule of the extension point of the grid structure. This generation rule is determined by which triangle in the configuration file is matched with which two known points of the face (these two points are AI points or the extension points that have been calculated), you can An extension point of the grid structure to be constructed is calculated.

Thirdly, the image processing apparatus can perform operations by using the similar triangle algorithm according to the configuration file, and calculate the coordinate value of the extension point. As shown in Figure 16, here the algorithm will use 5 known points, three of which are the points in the aforementioned configuration file (usually two AI points, one extension point), here named A, B, C (that is, A and B are AI points, and C is an extension point). The other two points are provided by the AI module, and their values are the values of screen coordinates, which are named A` and B` here. In the grid structure, A and A`, B and B` represent the same point. The value of a point C` can be obtained through the knowledge of basic trigonometric functions, which satisfies △ABC∽△A`B`C`, where C` is an extension point of the grid structure to be constructed.

Finally, by repeatedly calling the similar triangle algorithm many times, all the point information of the entire grid structure of the part to be beautified can be obtained.

Referring to FIG. 17 , an image processing apparatus provided by an embodiment of the present disclosure is shown. The image processing apparatus 170 may include:

The image acquisition module 1701 is configured to acquire a target image including the target cosmetic part;

The coordinate value obtaining module 1702 is configured to obtain the coordinate value of the contour feature point of the target beautified part in the target image, as the target coordinate value;

The grid structure acquisition module 1703 is configured to obtain the grid structure of the target beautified part according to the target coordinate value and a predetermined configuration file, as the target grid structure;

The drawing module 1704 is configured to, according to the target grid structure, perform beauty makeup processing on the beautified part in the target image to obtain a beauty makeup image;

In some embodiments, the grid structure acquisition module 1703 includes:

The first expansion point calculation sub-module 17031 is configured to calculate the coordinate value of the first type of expansion point according to the target coordinate value and the coordinate value of the vertex of the first type of triangle, wherein the first type of expansion point The expansion point is a point corresponding to the first expansion point in the target grid structure;

The grid structure drawing sub-module 17032 is configured to obtain the target grid structure according to the target coordinate value and the coordinate value of the first-type extension point.

The first extension point calculation sub-module 17031 is specifically configured as:

In some embodiments, the grid structure drawing sub-module 17032 includes:

The second extension point calculation unit 170321 is configured to calculate the coordinate value of the second type of extension point according to the coordinate value of the first type of extension point and the target coordinate value, wherein the second type of extension point is Other points in the target grid structure except the contour feature points of the part to be beautified in the target image and the extension points of the first type;

The grid structure drawing unit 170322 is configured to obtain the target grid structure according to the target coordinate value, the coordinate value of the first type of extension point and the coordinate value of the second type of extension point.

The second extension point calculation unit 170321 is specifically configured as:

In some embodiments, the second extension point calculation sub-module 17033 is specifically configured as:

It can be seen from the above that in the embodiment of the present disclosure, two contour feature points of the part to be beautified and a first extension point are used to form the first type of triangle, and used as the target triangle in the mesh structure of the part to be beautified, so that The coordinate values of these target triangles are formed into a configuration file, and then the configuration file and the coordinate values of the contour feature points of the target beautified part in the target image are used to calculate the target grid structure of the target beautified part, Further, according to the target grid structure, beauty makeup processing is performed on the target beauty makeup part to obtain a beauty makeup image.

Fig. 18 is a block diagram of an electronic device according to an exemplary embodiment. 18, the electronic device includes:

processor 1810;

memory 1820 for storing instructions executable by the processor;

Wherein, the processor is configured to execute the instructions to implement the image processing method described above.

FIG. 19 is a block diagram of another electronic device 1900 according to an exemplary embodiment. For example, electronic device 1900 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.

19, an electronic device 1900 may include one or more of the following components: a processing component 1902, a memory 1904, a power supply component 1906, a multimedia component 1908, an audio component 1910, an input/output (I/O) interface 1912, a sensor component 1914 , and the communication component 1916.

The processing component 1902 generally controls the overall operation of the electronic device 1900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1902 can include one or more processors 1920 to execute instructions to perform all or part of the steps of the methods described above. Additionally, processing component 1902 may include one or more modules that facilitate interaction between processing component 1902 and other components. For example, processing component 1902 may include a multimedia module to facilitate interaction between multimedia component 1908 and processing component 1902.

Memory 1904 is configured to store various types of data to support operation at electronic device 1900 . Examples of such data include instructions for any application or method operating on the electronic device 1900, contact data, phonebook data, messages, pictures, videos, and the like. Memory 1904 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 1906 provides power to various components of electronic device 1900. Power supply components 1906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1900 .

Multimedia component 1908 includes a screen that provides an output interface between the electronic device 1900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In the case where the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 1908 includes a front-facing camera and/or a rear-facing camera. When the electronic device 1900 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 1910 is configured to output and/or input audio signals. For example, audio component 1910 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 1900 is in operational modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 1904 or transmitted via communication component 1916. In some embodiments, audio component 1910 also includes a speaker for outputting audio signals.

The I/O interface 1912 provides an interface between the processing component 1902 and peripheral interface modules, which may be keyboards, click wheels, buttons, and the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 1914 includes one or more sensors for providing various aspects of status assessment for electronic device 1900 . For example, the sensor assembly 1914 can detect the on/off state of the electronic device 1900, the relative positioning of the components, such as the display and the keypad of the electronic device 1900, the sensor assembly 1914 can also detect the electronic device 1900 or one of the electronic device 1900 The location of components changes, the presence or absence of user contact with the electronic device 1900, the orientation or acceleration/deceleration of the electronic device 1900, and the temperature of the electronic device 1900 changes. Sensor assembly 1914 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 1914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 1916 is configured to facilitate wired or wireless communication between electronic device 1900 and other devices. The electronic device 1900 may access wireless networks based on communication standards, such as WiFi, carrier networks (eg, 2G, 3G, 19G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 1916 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 1900 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components are implemented for performing the image processing method described above.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as memory 1904 including instructions, executable by the processor 1920 of the electronic device 1900 to accomplish the above method. Alternatively, for example, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM , tapes, floppy disks and optical data storage devices.

In yet another aspect of the implementation of the present disclosure, an embodiment of the present disclosure further provides a storage medium, which enables the electronic device to execute the above-mentioned instructions when the instructions in the storage medium are executed by a processor of an electronic device. image processing method.

According to yet another aspect of the embodiments of the present disclosure, there is provided a computer program product including instructions, which enables the computer to implement the above-described image processing method when the computer program product runs on a computer.

All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification for the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

An image processing method, comprising:

Obtain the target image containing the target cosmetic part;

Obtain the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

According to the target coordinate value and the predetermined configuration file, the grid structure of the target cosmetic part is obtained as the target grid structure;

According to the target grid structure, beauty makeup processing is performed on the target beauty makeup part to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be beautified and coordinate values of vertices of the preset triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The vertex of the triangle includes two contour feature points of the part to be beautified and a first expansion point, and the first expansion point is a point set in advance according to the contour of the part to be beautified.
The image processing method according to claim 1, wherein, according to the target coordinate value and a predetermined configuration file, the grid structure of the target cosmetic part is obtained, comprising:

According to the target coordinate value and the coordinate value of the vertex of the first type of triangle, the coordinate value of the first type of extension point is calculated, wherein the first type of extension point is in the target mesh structure, and the point corresponding to the first extension point;

According to the target coordinate value and the coordinate value of the first type of extension point, the target grid structure is obtained.
The image processing method according to claim 2, wherein the target mesh structure includes at least one first target triangle, the first target triangle is in a one-to-one correspondence with the first type of triangle, and the first The class triangle is similar to its corresponding first target triangle; the first target triangle is in one-to-one correspondence with the first class extension point, and the first class extension point belongs to the vertex of the corresponding first target triangle;

According to the target coordinate value and the coordinate value of the vertex of the first-type triangle, the coordinate value of the first-type extension point is calculated and obtained, including:

According to the similarity relationship between the first type of triangle and its corresponding first target triangle, as well as the target coordinate value and the coordinate value of the vertex of the first type of triangle, the coordinates of the first type of extension point are calculated and obtained value.
The image processing method according to claim 2, wherein the obtaining the target grid structure according to the target coordinate value and the coordinate value of the first type of extension point comprises:

According to the coordinate value of the first type of extension point and the target coordinate value, the coordinate value of the second type of extension point is calculated, wherein the second type of extension point is the target grid structure divided by the target Contour feature points of the part to be beautified and points other than the first type of extension points;

The target grid structure is obtained according to the target coordinate value, the coordinate value of the first type extension point and the coordinate value of the second type extension point.
The image processing method according to claim 4, wherein the preset triangle further comprises at least one triangle of the second type, wherein a vertex of a triangle of the second type comprises a second extension point and two of the target set point, the target set includes the contour feature points of the part to be beautified and the first expansion point, and the second expansion point is a point set in advance according to the contour of the part to be beautified;

The target mesh structure also includes at least one second target triangle, the second target triangle is in one-to-one correspondence with the second type of triangle, and the second type of triangle is similar to its corresponding second target triangle , wherein the second target triangle is in one-to-one correspondence with the second type of extension point, and the second type of extension point belongs to the vertex of the corresponding second target triangle;

According to the coordinate value of the first type of expansion point and the target coordinate value, the calculation to obtain the coordinate value of the second type of expansion point, including:

According to the similarity relationship between the second type of triangle and its corresponding second target triangle, and the target coordinate value, the coordinate value of the first type of extension point, the coordinate value of the vertex of the first type of triangle and The coordinate value of the vertex of the second type of triangle is calculated to obtain the coordinate value of the second type of extension point.
The image processing method according to claim 4, wherein the calculating and obtaining the coordinate value of the second type extension point according to the coordinate value of the first type extension point and the target coordinate value comprises:

Using a smooth transition or interpolation method, the coordinate value of the second type of extension point is calculated according to the coordinate value of the first type of extension point and the target coordinate value.
An image processing device, comprising:

an image acquisition module, configured to acquire a target image containing the target cosmetic part;

A coordinate value acquisition module, configured to acquire the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

a grid structure acquisition module, configured to obtain the grid structure of the target cosmetic part according to the target coordinate value and a predetermined configuration file, as the target grid structure;

a drawing module, configured to perform beauty makeup processing on the beautified part in the target image according to the target grid structure to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be beautified and coordinate values of vertices of the preset triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The vertex of the triangle includes two contour feature points of the part to be beautified and a first expansion point, and the first expansion point is a point set in advance according to the contour of the part to be beautified.
The image processing apparatus according to claim 7, wherein the grid structure acquisition module comprises:

The first extension point calculation sub-module is configured to calculate the coordinate value of the first type of extension point according to the target coordinate value and the coordinate value of the vertex of the first type of triangle, wherein the first type of extension point The point is the point corresponding to the first expansion point in the target grid structure;

The grid structure drawing sub-module is configured to obtain the target grid structure according to the target coordinate value and the coordinate value of the first-type extension point.
The image processing apparatus according to claim 8, wherein the target mesh structure includes at least one first target triangle, the first target triangle is in one-to-one correspondence with the first type of triangle, and the first target triangle is in a one-to-one correspondence. The class triangle is similar to its corresponding first target triangle; the first target triangle is in one-to-one correspondence with the first class extension point, and the first class extension point belongs to the vertex of the corresponding first target triangle;

The first extension point calculation submodule is specifically configured as:

According to the similarity relationship between the first type of triangle and its corresponding first target triangle, as well as the target coordinate value and the coordinate value of the vertex of the first type of triangle, the coordinates of the first type of extension point are calculated and obtained value.
The image processing apparatus according to claim 8, wherein the grid structure drawing sub-module comprises:

The second extension point calculation unit is configured to calculate the coordinate value of the second type of extension point according to the coordinate value of the first type of extension point and the target coordinate value, wherein the second type of extension point is the Other points in the target grid structure except the contour feature points of the target beautified part and the first type of extension points;

The grid structure drawing unit is configured to obtain the target grid structure according to the target coordinate value, the coordinate value of the first type of extension point and the coordinate value of the second type of extension point.
The image processing apparatus according to claim 10, wherein the preset triangle further includes at least one second type triangle, wherein a vertex of one second type triangle includes a second extension point and two of the target set point, the target set includes the contour feature points of the part to be beautified and the first expansion point, and the second expansion point is a point set in advance according to the contour of the part to be beautified;

The target mesh structure also includes at least one second target triangle, the second target triangle is in one-to-one correspondence with the second type of triangle, and the second type of triangle is similar to its corresponding second target triangle , wherein the second target triangle is in one-to-one correspondence with the second type of extension point, and the second type of extension point belongs to the vertex of the second target triangle corresponding to it;

The second extension point calculation unit is specifically configured as:

According to the similarity relationship between the second type of triangle and its corresponding second target triangle, and the target coordinate value, the coordinate value of the first type of extension point, the coordinate value of the vertex of the first type of triangle and The coordinate value of the vertex of the second type of triangle is calculated to obtain the coordinate value of the second type of extension point.
The image processing apparatus according to claim 10, wherein the second extension point calculation submodule is specifically configured as:

Using a smooth transition or interpolation method, the coordinate value of the second type of expansion point is calculated according to the coordinate value of the first type of expansion point and the target coordinate value.
An electronic device comprising:

processor;

memory for storing instructions executable by the processor;

Wherein, the processor is configured to execute the instructions to implement the following steps:

Obtain the target image containing the target cosmetic part;

Obtain the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

According to the target coordinate value and the predetermined configuration file, the grid structure of the target cosmetic part is obtained as the target grid structure;

According to the target grid structure, beauty makeup processing is performed on the target beauty makeup part to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be beautified and coordinate values of vertices of the preset triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The vertex of the triangle includes two contour feature points of the part to be beautified and a first extension point, and the first extension point is a point set in advance according to the contour of the part to be beautified.
A non-volatile storage medium, wherein, when the instructions in the storage medium are executed by a processor of an electronic device, the following steps are implemented:

Obtain the target image containing the target cosmetic part;

Obtain the coordinate value of the contour feature point of the target cosmetic part in the target image, as the target coordinate value;

According to the target coordinate value and the predetermined configuration file, the grid structure of the target cosmetic part is obtained as the target grid structure;

According to the target grid structure, beauty makeup processing is performed on the target beauty makeup part to obtain a beauty makeup image;

Wherein, the configuration file includes preset triangles in the mesh structure of the part to be beautified and coordinate values of vertices of the preset triangles, and the preset triangles include at least one first-type triangle, one first-type triangle The vertex of the triangle includes two contour feature points of the part to be beautified and a first extension point, and the first extension point is a point set in advance according to the contour of the part to be beautified.