CN111652024B

CN111652024B - Face display and live broadcast method and device, electronic equipment and storage medium

Info

Publication number: CN111652024B
Application number: CN201910604515.5A
Authority: CN
Inventors: 华路延; 王云刚
Original assignee: Guangzhou Huya Technology Co Ltd
Current assignee: Guangzhou Huya Technology Co Ltd
Priority date: 2019-07-05
Filing date: 2019-07-05
Publication date: 2023-09-01
Anticipated expiration: 2039-07-05
Also published as: CN111652024A

Abstract

The embodiment of the invention discloses a method and a device for displaying and directly broadcasting a face, electronic equipment and a storage medium, wherein the method for displaying the face comprises the following steps: acquiring original image data, wherein the original image data has original face data; performing aging treatment on the original face data to obtain target face data; dividing the original image data into a plurality of first grids by taking first face key points of the original face data as at least partial vertexes of the first grids; dividing the target image data into a plurality of second grids by taking second face key points of the target face data as vertexes of the second grids; displaying target image data, the target image data comprising a partial texture of the first mesh and a partial texture of the second mesh. The texture is limited by the grids, so that the texture of the target face data is attached to the texture of the original image data, the situation that the face and the background are segmented is avoided, and the picture distortion is avoided.

Description

Face display and live broadcast method and device, electronic equipment and storage medium

Technical Field

The embodiment of the invention relates to an image processing technology, in particular to a method and a device for displaying and directly broadcasting a face, electronic equipment and a storage medium.

Background

With the development of society, electronic devices such as mobile phones and tablet computers have been widely used in learning, entertainment, work and other fields, and play an increasingly important role.

Cameras are configured in many electronic devices, and can be used for photographing, video recording, live broadcasting and other operations.

In applications such as AR (Augmented Reality ), making expressions, etc., the face of the current user is recognized, the age of the face representation is changed, and the changed face is replaced with the face before the change.

However, the face after the change is not necessarily matched with the face before the change, and the face and the background are easily separated after the replacement, so that the picture is distorted.

Disclosure of Invention

The embodiment of the invention provides a face display and live broadcast method, a device, electronic equipment and a storage medium, which are used for solving the problem that the situation that the face and the environment are easily segmented by replacing the face with the age-changed face, so that the picture is distorted.

In a first aspect, an embodiment of the present invention provides a method for displaying a face, including:

Acquiring original image data, wherein the original image data has original face data;

performing aging treatment on the original face data to obtain target face data;

dividing the original image data into a plurality of first grids by taking first face key points of the original face data as at least partial vertexes of the first grids;

dividing the target image data into a plurality of second grids by taking second face key points of the target face data as vertexes of the second grids;

displaying target image data, the target image data comprising a partial texture of the first mesh and a partial texture of the second mesh.

In a second aspect, an embodiment of the present invention further provides a live broadcast method, including:

collecting original video data, wherein the original video data is provided with original image data, and the original image data is provided with original face data;

Displaying target image data, the target image data comprising a partial texture of the first mesh and a partial texture of the second mesh;

and distributing target video data containing the target image data among the anytime broadcasting.

In a third aspect, an embodiment of the present invention further provides a display apparatus for a face, including:

the device comprises an original image data acquisition module, a storage module and a storage module, wherein the original image data acquisition module is used for acquiring original image data, and the original image data has original face data;

the aging processing module is used for performing aging processing on the original face data to obtain target face data;

the first grid dividing module is used for dividing the original image data into a plurality of first grids by taking first face key points of the original face data as at least partial vertexes of the first grids;

the second grid dividing module is used for dividing the target image data into a plurality of second grids by taking second face key points of the target face data as vertexes of the second grids;

and the target image data display module is used for displaying target image data, wherein the target image data comprises partial textures of the first grid and partial textures of the second grid.

In a fourth aspect, an embodiment of the present invention further provides a live broadcast apparatus, including:

the system comprises an original video data acquisition module, a storage module and a storage module, wherein the original video data acquisition module is used for acquiring original video data, the original video data are provided with original image data, and the original image data are provided with original face data;

a target image data display module, configured to display target image data, where the target image data includes a partial texture of the first mesh and a partial texture of the second mesh;

and the video live broadcast module is used for distributing target video data containing the target image data among the live broadcast.

In a fifth aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:

One or more processors;

a memory for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of displaying a face as described in the first aspect or the method of directly broadcasting as described in the second aspect.

In a sixth aspect, an embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, wherein the program when executed by a processor implements the method for displaying a face according to the first aspect or the method for directly broadcasting according to the second aspect.

In the embodiment of the invention, the original image data is acquired, the age represented by the original face data is regulated, the target face data is obtained, the original image data is divided into a plurality of first grids, the target face data is divided into a plurality of second grids, the target image data containing partial textures of the first grids and partial textures of the second grids is displayed, when the key points of the face are regulated, the grids can be regulated together, the regulation of the pixel points in the grids is uniform, so that the regulation of the face data is smoother, the deformation condition is reduced, the face distortion is avoided, the textures of the target face data are limited by the grids, the lamination of the textures of the target face data and the original image data is ensured, the situation of dividing the face and the background is avoided, and the picture distortion is avoided.

In addition, the grid can be simultaneously used for face adjustment and rendering operation, multiplexing grid division operation, reducing the operation amount, and the drawing and adjustment of the grid are simple, improving the processing speed and reducing the processing time, and is suitable for scenes with high requirements on real-time, such as live broadcasting.

Drawings

Fig. 1 is a flowchart of a face display method according to an embodiment of the present invention;

fig. 2A to fig. 2C are exemplary diagrams of a face key point according to a first embodiment of the present invention;

FIGS. 3A-3C are exemplary diagrams of a mesh sequence provided in accordance with a first embodiment of the present invention;

fig. 4 is an exemplary diagram of neighboring face key points according to a first embodiment of the present invention;

FIG. 5 is an exemplary diagram of a Veno diagram provided in accordance with a first embodiment of the present invention;

FIG. 6 is an exemplary diagram of a grid provided in accordance with one embodiment of the present invention;

fig. 7A to 7C are exemplary diagrams illustrating face aging according to a first embodiment of the present invention;

fig. 8 is a flowchart of a live broadcast method according to a second embodiment of the present invention;

fig. 9 is a schematic structural diagram of a face display device according to a third embodiment of the present invention;

fig. 10 is a schematic structural diagram of a live broadcast device according to a fourth embodiment of the present invention;

Fig. 11 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.

Example 1

Fig. 1 is a flowchart of a face display method according to an embodiment of the present invention, where the face display method is applicable to a case of constructing a grid based on face key points and driving a face based on the grid, and the method may be performed by a face driving display device, which may be implemented by software and/or hardware, and may be configured in an electronic device, and may include a mobile terminal such as a mobile phone, a tablet, a PDA (personal digital assistant, a personal digital assistant), a smart wearable device (such as a smart glasses and a smart watch), and may also include a non-mobile terminal such as a smart television and a personal computer, and the electronic device includes a processor such as a CPU (Central Processing Unit, a central processing unit), a GPU (Graphics Processing Unit, and a graphics processor), and an Application Programming Interface (API) or a rendering engine configured to render 2D vector graphics, such as OpenGL (Open Graphics Library ), openGL ES (OpenGL for Embedded Systems, openGL for embedded systems), metal, valkan, U D, UE4, and the method specifically includes the following steps:

S101, acquiring original image data.

In particular implementations, the operating system of the electronic device may include Android (Android), IOS, windows, and the like.

Applications capable of image processing, such as live broadcast applications, image editing applications, camera applications, instant messaging tools, gallery applications, and the like, are supported in these operating systems.

Applications such as image editing applications, instant messaging tools, gallery applications, etc., the UI (User Interface) of which may provide an imported control, a User may operate the imported control through a peripheral such as touch or mouse, select locally stored image data (represented by a thumbnail or path), and may also select network stored image data (represented by URL (Uniform Resource Locators, uniform resource locator)) so that the application obtains the image data as original image data.

Applications such as a live broadcast application, an image editing application, a camera application, an instant messaging tool and the like, a UI of the application can provide a control for photographing and video recording, a user can operate the control for photographing and video recording through peripheral equipment such as touch control or a mouse and the like, and the application is informed to call a camera to collect image data as original image data.

S102, aging processing is carried out on the original face data, and target face data are obtained.

The original image data has original face data, which may refer to pixels representing a face in the original image data.

The UI of the application may provide a control for the aging process, and the user may trigger the control, and may trigger an adjustment operation, in response to which the application may determine a span of age aging (e.g., 20 years of aging), and adjust the original face data in that direction to target face data after the span (e.g., face data after 20 years of aging).

In one implementation, a neural network may be determined that is used for the aging process.

The original face data is input into a neural network for aging treatment so as to output target face data

In this implementation, deep learning may be utilized to adjust the age of the original face data representation to achieve the aging process.

For example, in a CAAE (conditional adversarial autoencoder) network, a manifoldback automatic encoder can learn the manifold of face data, and predict the full-aged face image of the face data.

Furthermore, the capability of the CAAE network for adjusting the face representation age, namely the texture on the face data, can be reserved, so that the processing speed is improved.

S103, taking first face key points of the original face data as at least partial vertexes of a first grid, and dividing the original image data into a plurality of first grids.

In the embodiment of the invention, the face detection is carried out on the original image data, and the first face key points contained in the original face data are identified.

Face detection is also called face key point detection, positioning or face alignment, and refers to positioning key area positions of faces of a given face, including eyebrows, eyes, nose, mouth, face contours and the like.

Face detection generally uses the following method:

1. manually extracting features such as haar features, training a classifier by using the features, and performing face detection by using the classifier.

2. Face detection is inherited from a generic target detection algorithm, for example, using Faster R-CNN to detect faces.

3. Convolutional neural networks of Cascade structure are used, for example, cascade CNN (Cascade convolutional neural network), MTCNN (Multi-task Cascaded Convolutional Networks, multitasking convolutional neural network).

In a specific implementation, the methods for implementing face detection may be integrated in a module of an application, the application may directly call the module to detect a first face key point in the image data, the methods for implementing face detection may also be integrated in an SDK (Software Development Kit ), the SDK is used as assembly data of the application, the application may request the SDK to perform face detection on the original image data, the SDK detects the first face key point in the first image data, and returns the first face key point to the application.

It should be noted that, the number of the first face key points can be set by a person skilled in the art according to actual conditions, the real-time requirement for static image processing is low, and the denser first face key points, such as 1000, can be detected, and besides the important feature points of the face can be positioned, the outline of the five sense organs can be accurately described; for live broadcast and the like, the real-time requirements are high, sparse first face key points such as 68, 81 and 106 can be detected, and obvious and important characteristic points (such as eye key points, eyebrow key points, nose key points, mouth key points, contour key points and the like) on the face can be positioned, so that the processing amount is reduced, the processing time is shortened, and the embodiment of the invention is not limited to the above.

In order to enable those skilled in the art to better understand the embodiments of the present invention, in the embodiments of the present invention, a sparse first face key point is taken as an example to describe the embodiments of the present invention.

For example, face detection is performed on the original image data shown in fig. 2A, and 68 first face key points as shown in fig. 2B may be output.

And the first grid (Mesh) represents a single renderable entity, the first vertex of the first Mesh at least comprises a first face key point, namely, the first face key point is used as at least part of the first vertex of the first grid, and the original image data is gridded and divided into a plurality of (two and more) first grids.

Gridding is the division of concave polygons or polygons intersected by edges into convex polygons, such as triangles, for rendering by an API or rendering engine such as OpenGL.

It should be noted that, the first grids are orderly arranged to form a grid sequence, so as to conform to the rendering specifications of the API or the rendering engine such as OpenGL.

For example, for OpenGL, there are typically three types of drawing a series of triangles (meshes):

1、GL_TRIANGLES

every three points are respectively provided with a group of triangles, and the triangles are independent.

As shown in FIG. 3A, the first triangle uses vertex v ₀ 、v ₁ 、v ₂ The second triangle uses vertex v ₃ 、v ₄ 、v ₅ And so on.

2、GL_TRIANGLE_STRIP

Starting from the third point, each point, in combination with the first two points, draws a triangle, i.e. a string of linear continuous triangles:

as shown in FIG. 3B, the first triangle has vertices arranged in order v ₀ ,v ₁ ,v ₂ The method comprises the steps of carrying out a first treatment on the surface of the The second triangle, vertex order v ₂ ,v ₁ ,v ₃ The method comprises the steps of carrying out a first treatment on the surface of the The third triangle, the vertex arrangement order is v ₂ ,v ₃ ,v ₄ The method comprises the steps of carrying out a first treatment on the surface of the Fourth triangle, vertex arrangement order is v ₄ ,v ₃ ,v ₈ 。

This order is to ensure that the triangles are drawn in the same direction so that the sequence of triangles correctly forms part of the surface.

3、GL_TRIANGLE_FAN。

Starting from the third point, each point in combination with the previous point and the first point draws a triangle, i.e. a sector of a continuous triangle.

As shown in FIG. 3C, the vertex order of the first triangle is v ₂ ,v ₁ ,v ₀ The vertex order of the second triangle is v ₃ ,v ₂ ,v ₀ The vertex order of the first triangle is v ₄ ,v ₃ ,v ₀ 。

In one embodiment of the present invention, S103 may include the steps of:

s1031, determining first face key points adjacent in position as first target key points.

S1032, taking the first target key point as a first vertex in the first original image data, and connecting the first target key point to obtain a first grid.

In the embodiment of the invention, for two adjacent first face key points, the two adjacent first face key points can be used as a pair of first target key points, and the first target key points are sequentially used as the first vertexes of the first grids, and the first grids can be generated by connecting the pair of first target key points.

In one approach to detecting the proximity of the positions, the original image data including the key points of the first face may be converted into a first dimension norgram (Voronoi diagram) by a definition method (Intersect of Halfplanes), an increment (increment) algorithm, a divide-and-conquer method, a plane sweep algorithm, or the like.

The first dimension of the norgram, also known as the Thiessen polygon or Dirichlet graph, comprises a set of consecutive polygons (also known as cell) made up of perpendicular bisectors connecting two adjacent points straight lines.

In the first dimension northgraph, the euclidean distance between any two first face keypoints p and q is denoted as dist (p, q).

Let p= { P ₁ ,p ₂ ,…,p _n And the first face key points are any n mutually different first face key points on the plane, namely base points. The so-called P-corresponding Voronoi diagram is a sub-region division of a plane-the whole plane is thus divided into n cells, which have the property:

any first face key q is located at the first face key p _i In the corresponding units, if and only if for any p _j ∈P _j J.noteq.i, all have dist (q, p _i )＜dist(q,p _j ). At this time, the Voronoi diagram corresponding to P is referred to as Vor (P).

"Vor (P)" or "Voronoi diagram" indicates the edges and vertices that make up the sub-region division. In Vor (P), with base point P _i The corresponding cell is denoted as V (p _i ) Called AND p _i Corresponding Voronoi cells.

In the embodiment of the invention, the first dimension norgram comprises a plurality of first units, each first unit comprises a first face key point, the first units are provided with a plurality of first edges, at the moment, the first face key points positioned on two sides of the same first edge can be determined to be adjacent, and the first face key points positioned on two sides of the same edge are connected, so that a first grid can be generated.

For example, referring to fig. 4, the original image data is converted into a first dimension norgram, and fig. 4 shows a portion of the first face key points (black dots) and a portion of the edges of the first cells (solid lines).

In the first unit 400, the first face keypoints 411 on both sides of the first edge 401 are adjacent to the first face keypoints 412, and the first face keypoints 411 and the first face keypoints 412 may be connected, and similarly, the first face keypoints 411 and the first face keypoints 413 are connected, and the first face keypoints 413 and the first face keypoints 412 are connected, so as to generate the first mesh (dotted line edge) 420.

Further, the original image data shown in fig. 2B, which includes 68 first face key points, may be converted into a first dimension chart as shown in fig. 5, so as to determine first face key points adjacent to each other, and connect the first face key points adjacent to each other according to a predetermined sequence, so as to obtain a first grid as shown in fig. 6.

In another embodiment of the present invention, S103 may include the steps of:

s1033, determining a point located on the edge of the original image data as an edge point.

S1034, determining a first face key point adjacent to the edge point position as a third target key point.

S1035, in the original image data, the edge point and the third target key point are used as first vertexes, and the edge point and the third target key point are connected to obtain a first grid.

The first grid of first face key points generally cannot completely cover the original image data, and at this time, some points may be selected on the edge of the original image data as edge points.

It should be noted that, for convenience of operation, the selected edge points are symmetrical.

For example, as shown in fig. 6, four vertices of the original image data, and a midpoint between every two vertices are selected as edge points.

And taking the edge points and the adjacent first face key points as a pair of third target key points, sequentially taking the third target key points as the first vertexes of the first grid, and sequentially connecting the pair of third target key points, so that the first grid can be generated.

It should be noted that, the adjacent relationship between the edge point and the first face key point may be set by those skilled in the art according to the actual situation, which is not limited in the embodiment of the present invention.

In general, the eyebrow key points and the contour key points in the first face key points are adjacent to the edge points, and the number of the first edge points is smaller than that of the first face key points adjacent to the first edge points.

For example, the first image data shown in fig. 2B includes 68 first face key points and 8 first edge points, and 25 first face key points are adjacent to 8 edge points, and the first face key points and the first edge points adjacent to each other are connected in a predetermined order, so that a first grid as shown in fig. 6 can be obtained.

It should be noted that, the first vertex of the first mesh may include both the first face key point and the first edge point, and the points adjacent to each first face key point and each edge point (may be the first face key point or the edge point) are determined, and at this time, the first face key points and the edge points may be connected in a predetermined order, so as to generate the first mesh.

In yet another embodiment of the present invention, the first vertex of the first mesh includes a first face key point, and an edge point located on an edge of the original image data, the edge point and the first face key point have a first number, wherein the first number of the edge point is preset, the first number of the first face key point is generated during face detection, or is obtained based on a number map generated during face detection, and the first number of the edge point and the first number of the first face key point are not repeated.

For example, 68 first face key points are shown in fig. 2C, where the case is as follows:

the number of the outline key points is 17, and the first numbers are respectively 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17.

The total of 10 eyebrow key points are respectively 18, 19, 20, 21, 22, 23, 24, 25, 26 and 27.

The number of nose key points is 9, and the first numbers are 28, 29, 30, 31, 32, 33, 34, 35 and 36 respectively.

The total number of the eye key points is 12, and the first numbers are 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47 and 48 respectively.

The number of the key points of the mouth is 20, and the first numbers are 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67 and 68 respectively.

The number of edge points is 8, and the first numbers are 69, 70, 71, 72, 73, 74, 75 and 76 respectively.

At this time, S103 may include the steps of:

s1036, inquiring the preset grid variables with sequences.

Wherein vertices in each mesh variable are marked with a third number.

And S1037, if the first number is the same as the third number, taking an edge point or a first face key point which the first number belongs to as a first vertex in the original image data, and connecting the edge point or the first face key point which the first number belongs to obtain a first grid.

Because the method for face detection is preset, the output first face key points are generally fixed, and the edge points are also generally fixed, so that the points (which may be the first face key points or the edge points) adjacent to each first face key point and each edge point are all fixed, that is, the order of the first vertices in each first grid is fixed.

Therefore, when dividing the original image data of a certain frame into a plurality of first grids in a certain way for the first time (offline), the number of each first vertex (which may be a first face key point or an edge point) in each first grid is sequentially recorded as a third number, and at this time, the third number of the first vertex is reserved as a grid variable for each first grid.

For other (real-time) original image data, if the first grid is divided in the same manner, the first number of the point (may be the first face key point or the edge point) and the third number of the grid variable may be matched, and if the first number and the third number are the same, the points (may be the first face key point or the edge point) indicated by the first number may be connected in the order defined in the grid variable, so that the original image data is divided into a plurality of first grids.

For example, as shown in fig. 2C, the grid variables may be represented as (1, 76,2), (2, 76,3), (3, 76,4), (4, 76,5), and so on.

For the original image data shown in fig. 2B, the first face key points with the first numbers 1 and 2 and the edge point with the first number 76 are successfully matched with the third number in one of the grid variables, and then the first face key points and the edge points can be connected in sequence, so as to form a first grid (1, 76,2).

S104, taking the first face key points of the original face data as at least partial vertexes of the first grids, and dividing the original image data into a plurality of first grids.

And carrying out face detection on the target face data by using the target face data, and identifying second face key points contained in the target face data.

It should be noted that, the face recognition mode of the target face data is consistent with that of the original image data, so as to ensure that the second face key points are consistent with the first face key points.

And the second grid (Mesh) represents a single entity capable of being drawn, and the second vertex at least comprises second face key points of the target face data, namely, the second face key points are used as at least part of second vertices of the second grid, and the second target face data is gridded and divided into a plurality of (two or more) second grids.

In one embodiment of the present invention, S104 includes:

determining second face key points adjacent to each other in position as second target key points;

and in the target face data, the second target key point is used as a second vertex and is connected with the second target key point to obtain a second grid.

Further, the determining the second face key point with adjacent positions as the second target key point includes:

converting the target face data into a second dimension norgram, wherein the second dimension norgram comprises a plurality of second units, each second unit comprises a second face key point, and the second units are provided with a plurality of second edges;

and determining that the second face key points positioned on two sides of the same second edge are adjacent.

In a specific implementation, the second face key point has a second number, and S104 includes:

inquiring preset grid variables with sequences, wherein the vertex in each grid variable is marked with a third number;

and if the second number is the same as the third number, a second face key point which the second number belongs to is used as a second vertex in the target face data, and the second face key point which the second number belongs to is connected to obtain a second grid.

It should be noted that, since the dividing manner of the second grid is consistent with the dividing manner of the first grid, the description of the dividing manner of the second grid is relatively simple, and the relevant points only need to be referred to in the part of the description of the dividing manner of the first grid, which is not described in detail herein.

S105, displaying target image data.

Texture is a visual feature reflecting a homogeneous phenomenon in an image that represents a slowly varying or periodically varying surface structure organization arrangement property of the object surface. Texture has three major markers:

some local sequence is repeated continuously;

non-random arrangement;

the texture region is substantially uniform.

While the texture embodies the nature of global features, it also describes the surface properties of the scene to which the image or image region corresponds. However, since texture is only a characteristic of the surface of an object, and cannot fully reflect the intrinsic properties of the object, high-level image contents cannot be obtained by using only texture features. Unlike color features, texture features are not pixel-based features, which require statistical calculations in areas containing multiple pixels. In pattern matching, the regional characteristics have great advantages, and the matching cannot be successfully performed due to local deviation.

In a specific implementation, on the basis of original image data, partial textures of a first grid are reserved, partial textures of a second grid are added, such as the textures of parts, such as cheeks, forehead, eye corners and the like, which are easy to characterize the age, so that target image data is generated, namely the target image data comprises the partial textures of the first grid and the partial textures of the second grid, so that the human face in the target image data keeps the connection between the original human face data and the background data, has the age characteristic of the target human face data, and realizes the effect of adjusting the age.

For example, the age, that is, aging, represented by the original face data in the original image data shown in fig. 7A is adjusted to obtain the target face data shown in fig. 7B, and the partial texture of the target face data shown in fig. 7B is added to obtain the target image data shown in fig. 7C on the basis of retaining the partial texture in the original image data shown in fig. 7A.

In one embodiment of the present invention, S105 includes:

s1051, adjusting the second face key point to enable the second grid to be aligned with the first grid.

In the embodiment of the invention, since the original face data and the target face data are not necessarily identical, the second face key point of the target face data can be adjusted based on the first face key point of the original face data, so that the second face key point is aligned with the first face key point.

While maintaining the second face keypoints as the second vertices of the second mesh, the second face keypoints may be adjusted, along with the second mesh, to align the first mesh such that the target face data within the second mesh is approximately the same as the contour of the original face data.

Further, the alignment may refer to determining the vertex coordinates of the first face key point, determining the vertex coordinates of the second face key point, and for the first face key point and the second face key point having the same number (i.e. the first number of the first face key point is the same as the second number of the second face key point), assigning the vertex coordinates of the first face key point to the vertex coordinates of the second face key point so that the vertex coordinates of the second face key point are the same as the vertex coordinates of the first face key point.

S1052, determining texture coordinates of each first face key point in the first grid.

S1053, determining texture coordinates of part of the second face key points in the second grid.

Texture images are square arrays, texture coordinates can be generally defined in one, two, three or four dimensional forms, called s, t, r and q coordinates, one-dimensional textures are often represented by s coordinates, two-dimensional textures are often represented by (s, t) coordinates, and r coordinates are currently ignored. The q coordinate is like w and is typically 1, and is mainly used for establishing homogeneous coordinates. The OpenGL coordinates define the functions:

void gltexCoord{1234}{sifd}[v](TYPE coords)；

The current texture coordinates are set, and vertices generated by calling glVertex () are given to the current texture coordinates. For gltexCoord1 (), s coordinates are set to given values, t and r are set to 0, q is set to 1; s and t coordinate values can be set with gltexCoord2 (), r is set to 0, q is set to 1; for gltexCoord3 (), q is set to 1, and the other coordinates are set according to given values; all coordinates can be given by gltexCoord4 ().

S1054, assigning the texture coordinates of part of the second face key points to the texture coordinates of the first face key points.

In a specific implementation, for a first face key point and a second face key point (i.e., the first number of the first face key point is the same as the second number of the second face key point) with the same number for parts (such as cheeks, forehead, corners, etc. easy to characterize the age), the texture coordinates of the second face key point are assigned to the texture coordinates of the first face key point, so that the texture coordinates of the first face key point are the same as the texture coordinates of the second face key point.

Namely, the fixed point coordinates of part of the first face key points are bound with the texture coordinates of part of the second face data.

And S1054, drawing the first grid according to the texture coordinates so as to display target image data.

In the electronic equipment, an API or a rendering engine for rendering the 2D vector graphics is called, and each first grid is sequentially drawn, so that target image data is displayed on a screen, and the age change of original face data in the target image data along with the texture of the target face data can be displayed.

Further, to increase the display speed of the target image data, a first mesh may be drawn in the GPU.

In a specific implementation, S1054 includes:

s10541, for each first mesh, determining texture coordinates of each first vertex located in the first mesh.

S10542, for each first mesh, determining vertex coordinates of respective first vertices located in the first mesh.

And S10543, drawing the first grid according to the texture coordinates and the vertex coordinates so as to display target image data.

In rendering a texture mapped scene, in addition to defining geometric coordinates (i.e., vertex coordinates) for each vertex, texture coordinates are also defined. After various transformations, the geometric coordinates determine the location of the vertex drawn on the screen, while the texture coordinates determine which texel in the texture image is assigned to the vertex.

In the embodiment of the invention, the flow of drawing the grid is explained by taking OpenGL ES as an example, and the flow is a programmable pipeline and comprises the following operations:

1. VBO/VAO (Vertex Buffer/array Objects, vertex Buffer Objects or Vertex array Objects)

VBO/VAO is vertex information provided to the GPU by the CPU, including vertex coordinates, color (only the color of the vertex, independent of the color of the texture), texture coordinates (for texture mapping), and the like.

2. VertexSlader (vertex shader)

Vertex shader is a program that processes vertex information provided by VBO/VAO. Each vertex provided by VBO/VAO performs a pass through the vertex shader. The uniformity (a variable type) remains consistent at each vertex, with Attribute being different for each vertex (which can be understood as the input vertex attributes). The VertexShader is executed once to output a variable and gl_positon.

Wherein the vertex shader inputs include:

2.1, shader program: vertex shader program source code or executable file describing operations performed on vertices

2.2, vertex shader input (or attribute): data for each vertex provided by a vertex array

2.3, unified variable (unitorm): invariant data for vertex/fragment shader usage

2.4, samplers (Samplers): special unified variable types representing textures used by vertex shaders

Wherein, the vertex shader is a programmable stage of the vertex shader, which is used to control the conversion process of vertex coordinates, and the fragment shader controls the calculation process of each pixel color.

3. Primitive Assembly (primitive assembly):

the next stage of the vertex shader is primitive assembly, where primitives (primitives) are geometric objects such as triangles, lines, or points. At this stage, vertices output by the vertex shader are combined into primitives.

And restoring the vertex data into a grid structure according to the primary (original link relation), wherein the grid consists of vertexes and indexes, linking the vertexes together according to the indexes at the stage to form three different primitives of points, lines and planes, and then cutting triangles beyond a screen.

For example, if a triangle (mesh) has three vertices, one of which is outside the screen and the other two of which are inside the screen, and a quadrilateral is supposed to be seen on the screen, then the quadrilateral may be cut into 2 small triangles (meshes).

In short, the points obtained after the vertex shader computation are grouped into points, lines, planes (triangles) according to the link relationship.

4. rasterization (rasterization)

Rasterization is the process of converting a primitive into a set of two-dimensional fragments, which are then processed by a fragment shader (the input of the fragment shader). These two-dimensional segments represent pixels that can be rendered on a screen, and the mechanism for generating each segment value from the vertex shader output assigned to each primitive vertex is called interpolation.

The vertex after primitive assembly can be understood as a graph, and pixels (texture coordinates v_texcoord, color, and the like) of the graph area can be interpolated according to the shape of the graph during rasterization. Note that the pixel at this time is not a pixel on the screen, and is not colored. The next fragment shader performs the coloring.

5. FragmentSlader (fragment shader)

The fragment shader implements a generic programmable method for operations on fragments (pixels), with each fragment of the rasterized output executing a pass of the fragment shader, with each fragment generated by the rasterization stage executing this shader, generating one or more (multiple rendered) color values as output.

6. Per-Fragment Operations (fragment by fragment operation)

At this stage, each segment performs the following 5 operations:

6.1, pixelOwnershipTest (pixel home test)

It is determined whether the pixel at position (x, y) in the frame buffer is owned by the current context.

For example, if one display frame buffer window is obscured by another window, the windowing system may determine that the obscured pixels do not belong to the context of this OpenGL and thus do not display those pixels.

6.2, scissorTest (cut test):

if the segment is outside the clipping region, it is discarded.

6.3, stencilTest and DepthTest (template and depth test):

if the shape returned by the fragment shader is not a shape in the stencil, then it is discarded.

If the depth returned by the fragment shader is less than the depth in the buffer, then it is discarded.

6.4, blending (mixing):

the newly generated fragment color values are combined with the color values stored in the frame buffer to produce new RGBA (Red, green, blue and Alpha color spaces).

6.5, dithering (jitter):

at the end of the fragment-by-fragment phase of operation, fragments are either rejected or the color, depth or template value of the fragment is written at a certain position in the frame buffer (x, y). The write fragment color, depth, and template value depend on the corresponding write mask that is discarded. The write mask may more precisely control the color, depth, and stencil value of the write-related buffers. For example: the write mask of the color buffer may be set such that no red value can be written to the color buffer.

Finally, the generated fragment is placed in a frame buffer (front buffer or back buffer or FBO (Frame Buffer Object, frame buffer object)), and if not, the fragment in the screen rendering buffer generates pixels on the screen.

In another embodiment of the present invention, S105 includes:

s1055, adjusting the second face key point to enable the second grid to be aligned with the first grid.

S1056, drawing the first grid to display the original image data.

And S1057, drawing part of the second grid to display part of the target face data on the original image data, so as to obtain target image data.

In the embodiment of the invention, on the basis of drawing the original image data, the target face data of the selected part (such as cheek, forehead, canthus and other parts easy to characterize the age) is drawn on the corresponding position of the original image data, so that the target image data is displayed.

Further, in order to naturally attach the texture characterizing the age change to the original image data, a fusion algorithm may be used to fuse the original image data with a portion of the target face.

As an example of the fusion algorithm, poisson Image Editing may enable the attached partial target face data to be seamlessly fused into the original image data of the background by retaining the shaving degree.

In a specific implementation, S1056 includes:

for each first mesh, determining texture coordinates of respective first vertices located in the first mesh.

For each first mesh, vertex coordinates of respective first vertices located in the first mesh are determined.

And drawing the first grid according to the texture coordinates and the vertex coordinates so as to display the original image data.

S1057 includes:

for a portion of the second mesh, texture coordinates of each second vertex located in the second mesh are determined.

For a portion of the second mesh, vertex coordinates of each second vertex located in the second mesh are determined.

And drawing part of the second grid according to the texture coordinates and the vertex coordinates so as to display part of the target face data on the original image data, thereby obtaining target image data.

The drawing methods of the first and second grids are consistent with the drawing method of the first grid with the texture coordinates changed, so that the description of the drawing methods of the first and second grids is relatively simple, and the description of the drawing methods of the first grid with the texture coordinates changed is only needed to be referred to in the section of the description of the drawing methods of the first grid with the texture coordinates changed.

Example two

Fig. 8 is a flowchart of a live broadcasting method provided in a second embodiment of the present invention, where the method may be applied to a situation that a grid is built based on key points of a face, so as to replace the face for live broadcasting, and the method may be performed by a live broadcasting device, where the device may be implemented by software and/or hardware, may be configured in an electronic device, may include a mobile terminal such as a mobile phone, a tablet, a PDA, a smart wearable device (such as a smart glasses and a smart watch), and may also include a non-mobile terminal such as a smart television and a personal computer, and the electronic device includes a processor such as a CPU and a GPU, and is configured with an Application Programming Interface (API) or a rendering engine for rendering 2D vector graphics, and OpenGL, openGL ES, and the method specifically includes the following steps:

S801, collecting original video data.

Live applications are supported in these operating systems, which can perform image processing.

It should be noted that the live application may be an independent application, or may be a component in other applications (such as a social application, an instant messaging tool, etc.), such as an SDK, which is not limited in the embodiment of the present invention.

And the anchor user logs in a user account to the live broadcast platform in the live broadcast application, and sends a live broadcast title to the live broadcast platform so as to generate a live broadcast room for the user account according to the live broadcast title.

In one aspect, the live application may invoke a camera of the electronic device to collect raw video data having a plurality of frames of raw image data therein, at least a portion of the raw image data having a host user therein.

S802, performing aging processing on the original face data to obtain target face data.

The original image data has original face data, and as the original face data has small change in a short time, the live broadcast application can adjust the age (aging treatment) represented by the original face data in one frame of original image data to obtain target face data, and then the target face data is attached to each frame of original image data in sequence to obtain multi-frame target image data.

In one embodiment of the present invention, S802 includes:

determining a neural network, the neural network being used for an aging process;

and inputting the original face data into the neural network for aging treatment so as to output target face data.

Of course, the live broadcast application may also adjust the age (aging process) of the original face data representation in each frame of original image data to obtain multi-frame target face data, and then attach the multi-frame target face data to the corresponding multi-frame original image data to obtain multi-frame target image data.

S803, taking the first face key points of the original face data as at least partial vertexes of a first grid, and dividing the original image data into a plurality of first grids.

The first vertices of the first mesh include at least first face keypoints of the original face data.

In one embodiment of the present invention, S803 includes:

determining first face key points adjacent in position as first target key points;

and in the original image data, the first target key point is used as a first vertex and connected with the first target key point to obtain a first grid.

Further, the determining the first face key point with adjacent positions as the first target key point includes:

Converting the original image data into a first dimension norgram, wherein the first dimension norgram comprises a plurality of first units, each first unit comprises a first face key point, and the first units are provided with a plurality of first edges;

and determining that the first face key points positioned on two sides of the same first edge are adjacent.

In another embodiment of the present invention, S802 includes:

determining a point located on an edge of the original image data as an edge point;

determining a first face key point adjacent to the edge point position as a third target key point;

and in the original image data, the edge point and the third target key point are used as second vertexes, and the edge point and the third target key point are connected to obtain a first grid.

In practical application, the first vertex of the first mesh further includes an edge point located on the edge of the original image data, where the edge point and the first face key point have a first number, and S803 includes:

and if the first number is the same as the third number, taking an edge point or a first face key point to which the first number belongs in the original image data as a first vertex, and connecting the edge point or the first face key point to which the first number belongs to obtain a first grid.

In the embodiment of the present invention, since the dividing manner of the first grid is substantially similar to that of the first embodiment, the description is relatively simple, and the relevant points are only referred to in the description of the first embodiment, which is not described in detail herein.

S804, taking the second face key points of the target face data as the vertexes of the second grids, and dividing the target image data into a plurality of second grids.

The second vertex of the second mesh is a second face key point of the target face data.

In one embodiment of the present invention, S804 includes:

In practical application, the second face key point has a second number, and S804 includes:

In the embodiment of the present invention, since the dividing manner of the second grid is substantially similar to that of the first embodiment, the description is relatively simple, and the relevant points are only referred to in the description of the first embodiment, which is not described in detail herein.

S805, displaying target image data.

Wherein the target image data comprises a partial texture of the first mesh and a partial texture of the second mesh.

In practical application, an API or a rendering engine such as OpenGL, openGL ES and the like is called, each grid (a first grid and/or a second grid) is drawn in sequence, after the grids are drawn, target image data after the age represented by the adjustment face of the anchor user is displayed on a screen, and local back display of live broadcast (video data) is realized, so that the anchor user can browse the current live broadcast state.

In one embodiment of the present invention, S805 includes:

determining texture coordinates of each first face key point in the first grid;

determining texture coordinates of part of second face key points in the second grid;

assigning texture coordinates of part of the second face key points to the texture coordinates of the first face key points;

and drawing the first grid according to the texture coordinates so as to display target image data.

Further, S805 further includes:

and adjusting the second face key points to enable the second grid to be aligned with the first grid.

In another embodiment of the present invention, S805 includes:

drawing the first grid to display the original image data;

and drawing part of the second grid to display part of the target face data on the original image data so as to obtain target image data.

Further, S805 further includes:

In the embodiment of the present invention, since the display of the target image data is substantially similar to the application of the first embodiment, the description is relatively simple, and the relevant points are only referred to in the description of the first embodiment, and the embodiment of the present invention is not described in detail herein.

S806, distributing the target video data containing the target image data between the anytime broadcasting.

In a specific implementation, the live broadcast application packages the target image data and the audio data into target video data, and sends the target video data to a live broadcast platform (live broadcast server) through RTSP (Real Time Streaming Protocol, real-time streaming protocol), RTMP (Real Time Messaging Protocol, real-time message transmission protocol), HLS (HTTP Live Streaming, dynamic code rate adaptive technology) and other protocols, and the live broadcast platform (live broadcast server) discloses the live broadcast room, so that a viewer user can open the live broadcast room in a client, receive and play the target video data, and display the target image data after the host user adjusts the age represented by the face.

In the embodiment of the invention, the original video data is collected, the age represented by the original face data is regulated to obtain the target face data, the original image data is divided into a plurality of first grids, the target face data is divided into a plurality of second grids, the target image data containing partial textures of the first grids and partial textures of the second grids is displayed, the target video data containing the target image data is released in a direct broadcasting way, when the key points of the face are regulated, the grids can be regulated together, the regulation of the pixel points in the grids is uniform, so that the regulation of the face data is smoother, the deformation condition is reduced, the face distortion is avoided, the textures of the target face data are ensured to be attached to the textures of the original image data by limiting the textures by the grids, the situation that the face and the background are divided is avoided, and the picture distortion is avoided.

In addition, the grid can be simultaneously used for face adjustment and rendering operation, multiplexing grid division operation, so that the operation amount can be reduced, the drawing and adjustment of the grid are simple, the processing speed can be improved, the processing time can be reduced, the real-time performance can be ensured, and the method is suitable for live broadcasting.

Example III

Fig. 9 is a schematic structural diagram of a face display device according to a third embodiment of the present invention, where the device may specifically include the following modules:

the original image data acquisition module 901 is configured to acquire original image data, where the original image data has original face data;

the aging processing module 902 is configured to perform aging processing on the original face data to obtain target face data;

a first mesh dividing module 903, configured to divide the original image data into a plurality of first meshes by using first face key points of the original face data as at least part of vertices of the first meshes;

a second mesh dividing module 904, configured to divide the target image data into a plurality of second meshes by using second face key points of the target face data as vertices of the second meshes;

a target image data display module 905, configured to display target image data, where the target image data includes a partial texture of the first mesh and a partial texture of the second mesh.

In one embodiment of the invention, the aging processing module 902 includes:

the neural network determining submodule is used for determining a neural network, and the neural network is used for aging processing;

and the neural network processing sub-module is used for inputting the original face data into the neural network for aging processing so as to output target face data.

In one embodiment of the present invention, the target image data display module 905 includes:

the first texture coordinate determining submodule is used for determining texture coordinates of each first face key point in the first grid;

a second texture coordinate determining sub-module, configured to determine texture coordinates of a portion of second face key points in the second grid;

the texture coordinate assignment sub-module is used for assigning the texture coordinates of part of the second face key points to the texture coordinates of the first face key points;

and the texture drawing sub-module is used for drawing the first grid according to the texture coordinates so as to display target image data.

In another embodiment of the present invention, the target image data display module 905 includes:

a first grid drawing sub-module for drawing the first grid to display the original image data;

And the second grid drawing sub-module is used for drawing part of the second grid so as to display part of the target face data on the original image data and obtain target image data.

Further, the target image data display module 905 further includes:

In one embodiment of the present invention, the first meshing module 903 includes:

the first target key point determining submodule is used for determining first face key points adjacent in position and used as first target key points;

the first connection sub-module is used for taking the first target key point as a first vertex in the original image data and connecting the first target key point to obtain a first grid;

the second meshing module 904 includes:

the second target key point determining submodule is used for determining second face key points adjacent in position and used as second target key points;

and the second connection sub-module is used for taking the second target key point as a second vertex in the target face data and connecting the second target key point to obtain a second grid.

In one embodiment of the present invention, the first target keypoint determination submodule includes:

a first dimension norgram conversion unit, configured to convert the original image data into a first dimension norgram, where the first dimension norgram includes a plurality of first units, each first unit includes a first face key point, and the first units have a plurality of first edges;

the first position adjacent determining unit is used for determining that the first face key points positioned on two sides of the same first edge are adjacent;

the second target key point determining submodule includes:

the second dimension norgram conversion unit is used for converting the target face data into a second dimension norgram, the second dimension norgram comprises a plurality of second units, each second unit comprises a second face key point, and the second units are provided with a plurality of second edges;

and the second position adjacent determining unit is used for determining that the second face key points positioned on two sides of the same second edge are adjacent.

In another embodiment of the present invention, the first meshing module 903 includes:

an edge point determination submodule for determining a point located on the edge of the original image data as an edge point;

A third target key point determining submodule, configured to determine a first face key point adjacent to the edge point, as a third target key point;

and the third connection sub-module is used for taking the edge point and the third target key point as a second vertex in the original image data and connecting the edge point and the third target key point to obtain a first grid.

In yet another embodiment of the present invention, the first vertex of the first mesh further includes an edge point located on the edge of the original image data, the edge point having a first number with the first face key point;

the second face key points are provided with second numbers;

the first meshing module 903 includes:

the first grid variable searching sub-module is used for searching preset grid variables with sequences, and the vertex in each grid variable is marked with a third number;

a fourth connection sub-module, configured to, if the first number is the same as the third number, obtain a first mesh in the original image data by using an edge point or a first face key point to which the first number belongs as a first vertex, and connecting the edge point or the first face key point to which the first number belongs;

The second meshing module 904 includes:

the second grid variable searching sub-module is used for inquiring preset grid variables with sequences, and the vertex in each grid variable is marked with a third number;

and a fifth connection sub-module, configured to, if the second number is the same as the third number, obtain a second mesh in the target face data by using a second face key point to which the second number belongs as a second vertex and connecting the second face key point to which the second number belongs.

The face display device provided by the embodiment of the invention can execute the face display method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example IV

Fig. 10 is a schematic structural diagram of a live broadcast device according to a third embodiment of the present invention, where the device may specifically include the following modules:

the original video data acquisition module 1001 is configured to acquire original video data, where the original video data has original image data, and the original image data has original face data;

the aging processing module 1002 is configured to perform aging processing on the original face data to obtain target face data;

A first mesh dividing module 1003, configured to divide the original image data into a plurality of first meshes by using first face key points of the original face data as at least part of vertices of the first meshes;

a second mesh dividing module 1004, configured to divide the target image data into a plurality of second meshes by using second face key points of the target face data as vertices of the second meshes;

a target image data display module 1005 for displaying target image data, the target image data comprising a partial texture of the first mesh and a partial texture of the second mesh;

a live video module 1006, configured to issue target video data including the target image data between live video.

In one embodiment of the invention, the aging processing module 1002 includes:

In one embodiment of the present invention, the target image data display module 1005 includes:

In another embodiment of the present invention, the target image data display module 1005 includes:

Further, the target image data display module 1005 further includes:

In one embodiment of the present invention, the first meshing module 1003 includes:

the second meshing module 904 includes:

The second target key point determining submodule includes:

In another embodiment of the present invention, the first meshing module 1003 includes:

The second face key points are provided with second numbers;

the first meshing module 1003 includes:

the second meshing module 1004 includes:

The live broadcast device provided by the embodiment of the invention can execute the live broadcast method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example five

Fig. 11 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention. As shown in fig. 11, the electronic apparatus includes a processor 1100, a memory 1101, a communication module 1102, an input device 1103, and an output device 1104; the number of processors 1100 in the electronic device may be one or more, one processor 1100 being taken as an example in fig. 11; the processor 1100, the memory 1101, the communication module 1102, the input device 1103 and the output device 1104 in the electronic device may be connected by a bus or other means, in fig. 11 by way of example.

The memory 1101 is a computer-readable storage medium, and can be used to store a software program, a computer-executable program, and modules, such as modules corresponding to a display method of a face in the present embodiment (for example, an original image data acquisition module 901, a senescence processing module 902, a first meshing module 903, a second meshing module 904, a target image data display module 905 in a display device of a face as shown in fig. 9) or modules corresponding to a live broadcast method in the present embodiment (for example, an original video data acquisition module 1001, a senescence processing module 1002, a first meshing module 1003, a second meshing module 1004, a target image data display module 1005, a video live broadcast module 1006 in a live broadcast device as shown in fig. 10). The processor 1100 executes various functional applications of the electronic device and data processing, that is, implements the above-described face display method or live broadcast method, by running software programs, instructions, and modules stored in the memory 1101.

The memory 1101 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the electronic device, etc. In addition, the memory 1101 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 1101 may further include memory remotely located relative to processor 1100, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

And the communication module 1102 is used for establishing connection with the display screen and realizing data interaction with the display screen. The input device 1103 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device.

The electronic device provided by the embodiment of the invention can execute the face display method or the live broadcast method provided by any embodiment of the invention, and particularly has corresponding functions and beneficial effects.

Example six

A sixth embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon.

In one case, the program when executed by the processor implements a method for displaying a face, the method comprising:

In another case, the program when executed by the processor implements a live method comprising:

Of course, the computer readable storage medium provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the face display method or the live broadcast method provided by any of the embodiments of the present invention.

From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.

It should be noted that, in the embodiment of the foregoing face display device or live broadcast device, each unit and module included are only divided according to the functional logic, but are not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.

Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims

1. A method for displaying a face, comprising:

displaying target image data, wherein the target image data comprises partial textures of the first grid and partial textures of the second grid, and the target image data is generated by reserving the partial textures of the first grid and adding the partial textures of the second grid, which are used for representing the part of the age, on the basis of the original image data.

2. The method according to claim 1, wherein the aging the raw face data to obtain target face data includes:

3. The method of claim 1, wherein displaying the target image data comprises:

Determining texture coordinates of each first face key point in the first grid;

4. The method of claim 1, wherein displaying the target image data comprises:

drawing the first grid to display the original image data;

5. The method of claim 3 or 4, wherein displaying the target image data further comprises:

6. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the dividing the original image data into a plurality of first grids by taking the first face key points of the original face data as at least part of vertexes of the first grids comprises the following steps:

the first target key point is used as a first vertex in the original image data, and the first target key point is connected to obtain a first grid;

the dividing the target image data into a plurality of second grids by using the second face key points of the target face data as the vertexes of the second grids includes:

7. The method of claim 6, wherein the step of providing the first layer comprises,

the determining the first face key points with adjacent positions as the first target key points comprises the following steps:

determining that first face key points positioned on two sides of the same first edge are adjacent;

the determining the second face key points adjacent to the positions as the second target key points comprises the following steps:

8. The method of claim 1, wherein the dividing the original image data into a plurality of first meshes with the first face keypoints of the original face data as at least part of vertices of the first meshes comprises:

9. The method according to claim 1 or 2 or 3 or 4 or 6 or 7 or 8,

the first vertex of the first grid further comprises an edge point positioned on the edge of the original image data, and the edge point and the first face key point are provided with a first number;

The second face key points are provided with second numbers;

if the first number is the same as the third number, using an edge point or a first face key point to which the first number belongs as a first vertex in the original image data, and connecting the edge point or the first face key point to which the first number belongs to obtain a first grid;

10. A live broadcast method, comprising:

displaying target image data, wherein the target image data comprises partial textures of the first grid and partial textures of the second grid, and the target image data is generated by reserving the partial textures of the first grid and adding the partial textures of the second grid, which are used for representing the part of the age, on the basis of the original image data;

11. A display device for a human face, comprising:

the second grid dividing module is used for dividing the target image data into a plurality of second grids by taking the second face key points of the target face data as the vertexes of the second grids;

the target image data display module is used for displaying target image data, wherein the target image data comprises partial textures of the first grid and partial textures of the second grid, and the target image data is generated by reserving the partial textures of the first grid and adding the partial textures of the second grid, which are used for representing the part of the age, on the basis of the original image data.

12. A live broadcast device, comprising:

the target image data display module is used for displaying target image data, wherein the target image data comprises partial textures of the first grid and partial textures of the second grid, the target image data is generated by reserving the partial textures of the first grid and adding the partial textures of the second grid, which are used for representing the part of the age, on the basis of the original image data;

13. An electronic device, the electronic device comprising:

one or more processors;

a memory for storing one or more programs;

The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of displaying a face as claimed in any one of claims 1-9 or the method of direct broadcasting as claimed in claim 10.

14. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements a method of displaying a face as claimed in any one of claims 1-9 or a live method as claimed in claim 10.