CN113822965A

CN113822965A - Image rendering processing method, device and equipment and computer storage medium

Info

Publication number: CN113822965A
Application number: CN202110765134.2A
Authority: CN
Inventors: 郑青青; 赵远远
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-07-07
Filing date: 2021-07-07
Publication date: 2021-12-21

Abstract

The application discloses an image rendering processing method, an image rendering processing device and a computer storage medium, and relates to the technical field of artificial intelligence, in the method, when virtual eyebrow make-up is carried out on an image to be processed, based on key points of an eyebrow region, a deformation region to be processed is determined, and scaling mapping processing is carried out on the deformation region to be processed, so that the eyebrow region is reduced, and then the eyebrow portion with a make-up effect is added subsequently, the eyebrow region can be well covered, so that the make-up effect is better, in addition, when material rendering is carried out, pixel mapping relations are established, so that independent and parallel rendering can be carried out among pixels, the image rendering efficiency is improved, and the real-time performance of video make-up is correspondingly improved.

Description

Image rendering processing method, device and equipment and computer storage medium

Technical Field

The application relates to the technical field of computers, in particular to the technical field of artificial intelligence, and provides an image rendering processing method, an image rendering processing device, image rendering processing equipment and a computer storage medium.

Background

At present, in a video processing scene, such as a short video scene or a live video scene, the requirement of automatic face makeup often exists, eyebrows are an indispensable part in makeup, and the makeup effect is directly influenced.

In the related art, the scheme that eyebrow makeup can be automatically performed through an augmented reality mode exists, but the scheme is that eyebrow templates are directly superposed in the eyebrow area of a human face, and the problem that the eyebrow templates are not matched with the original eyebrows possibly exists, so that the eyebrow templates cannot completely cover the original eyebrows, and the makeup effect is poor. Based on this, the user can only select a suitable eyebrow template according to the eyebrow condition of the user, and can cover the original eyebrow, and when the user wants to try different eyebrow types, the related technical scheme still has certain limitation, and the better makeup effect can not be displayed.

Disclosure of Invention

The embodiment of the application provides an image rendering processing method, device and equipment and a computer storage medium, which are used for improving the makeup effect and the image rendering efficiency after image rendering.

In one aspect, an image rendering processing method is provided, and the method includes:

carrying out face key point identification on a first image to be processed containing a face, and identifying a plurality of first key points corresponding to a first actual eyebrow area of the face from the first image to be processed;

determining a deformation region to be processed containing the first actual eyebrow region in the first image to be processed based on the plurality of first key points, and performing scaling mapping processing on each pixel point in the deformation region to be processed to obtain a second image to be processed containing a scaled second actual eyebrow region;

obtaining a target eyebrow template containing a virtual makeup effect, and establishing a one-to-one corresponding pixel mapping relation between the first actual eyebrow area and the simulated eyebrow area based on the first key points and second key points corresponding to the simulated eyebrow area in the eyebrow template;

based on the pixel mapping relation, extracting rendering materials of all pixel points from the simulated eyebrow area, and respectively performing fusion rendering processing on corresponding pixel points in the second image to be processed to obtain a target image.

In one aspect, an image rendering processing apparatus is provided, the apparatus including:

the face recognition unit is used for carrying out face key point recognition on a first image to be processed containing a face and recognizing a plurality of first key points corresponding to a first actual eyebrow area of the face from the first image to be processed;

a scaling mapping unit, configured to determine, based on the plurality of first key points, a to-be-processed deformation region including the first actual eyebrow region in the first to-be-processed image, and perform scaling mapping processing on each pixel point in the to-be-processed deformation region to obtain a second to-be-processed image including a scaled second actual eyebrow region;

the grid construction unit is used for obtaining a target eyebrow template containing a virtual makeup effect and establishing a one-to-one corresponding pixel mapping relation between the first actual eyebrow area and the simulated eyebrow area based on the first key points and the second key points corresponding to the simulated eyebrow area in the eyebrow template;

and the texture fitting unit is used for extracting rendering materials of all pixel points from the simulated eyebrow region based on the pixel mapping relation, and performing fusion rendering processing on corresponding pixel points in the second image to be processed to obtain a target image.

Optionally, the scaling mapping unit is specifically configured to:

determining a reference point of the deformation region to be processed in the first image to be processed based on the coordinate values of the plurality of first key points in the constructed reference coordinate system;

and determining the deformation area to be processed in the first image to be processed by taking the reference point as a center based on a set shape model.

Optionally, the scaling mapping unit is specifically configured to:

and determining pixel points corresponding to the mean value of the coordinate values corresponding to the first key points as the reference points.

Optionally, the scaling mapping unit is specifically configured to:

determining the distance between the reference point and each first key point, determining a first distance with the largest value from the obtained distances, and determining a semi-major axis based on the first distance;

selecting a second distance with a larger value from the distances between the reference point and the two closest first key points, and determining a semi-minor axis based on the second distance;

and determining an elliptical area surrounded by the semi-major axis and the semi-minor axis by taking the reference point as a center as the deformation area to be processed.

Optionally, the scaling mapping unit is specifically configured to:

determining a connecting line between the reference point and a first key point corresponding to the first distance, and an included angle value between a set coordinate axis in the reference coordinate system;

aiming at each pixel point in the image to be processed, the following operations are respectively executed:

for a pixel point, taking the reference point as a center, rotating the pixel point by the included angle value, and determining the distance between the position of the rotated pixel point and the reference point;

and if the distance between the position of the rotated pixel point and the reference point is smaller than a set distance threshold, determining the pixel point as the pixel point in the deformation area to be processed.

Optionally, the scaling mapping unit is specifically configured to:

determining the distance between the reference point and each first key point, and determining the distance with the largest value from the obtained distances;

and determining a circular area which takes the reference point as the center and the distance with the largest value as the radius as the deformation area to be processed.

Optionally, the scaling mapping unit is specifically configured to:

and respectively executing the following operations aiming at each pixel point in the deformation region to be processed to obtain the second image to be processed:

aiming at a pixel point, determining a coordinate value of the pixel point after zooming based on the coordinate value of the pixel point and a set zooming mapping relation; in the set scaling mapping relation, the scaling is in negative correlation with the distance between the reference points of the deformation area to be processed;

and updating the pixel point at the position of the zoomed coordinate value according to the pixel characteristic value of the pixel point.

Optionally, the grid construction unit is specifically configured to:

based on the plurality of first key points, carrying out gridding processing on the first actual eyebrow area to obtain a first grid structure; wherein the first plurality of key points are vertices of meshes in the first mesh structure, and one mesh corresponds to a sub-region of the first actual eyebrow region; and the number of the first and second groups,

based on the plurality of second key points, carrying out gridding processing on the simulated eyebrow area to obtain a second grid structure; wherein the plurality of second key points are vertices of meshes in the second mesh structure, one mesh corresponds to one sub-region of the simulated eyebrow region, and meshes in the first mesh structure correspond to meshes of the second mesh structure one to one;

and establishing the pixel mapping relation based on the grid corresponding relation between the first grid structure and the second grid structure.

Optionally, the grid construction unit is specifically configured to:

taking the plurality of first key points and at least one set auxiliary key point as vertexes, and carrying out gridding processing on the first actual eyebrow area based on a set gridding processing mode to obtain a first grid structure;

the gridding processing mode is determined based on a standard eyebrow area in a standard face image, and the standard face image is a face image which is obtained by averaging various types of face templates and is in an orthographic view state.

Optionally, the apparatus further includes a video stream processing unit, configured to:

acquiring a video stream to be processed, and selecting an image containing a human face from the video stream to be processed as the image to be processed; and obtaining a target video stream containing the virtual makeup effect based on the target image corresponding to each image to be processed.

In one aspect, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the above methods when executing the computer program.

In one aspect, a computer storage medium is provided having computer program instructions stored thereon that, when executed by a processor, implement the steps of any of the above-described methods.

In one aspect, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps of any of the methods described above.

In the embodiment of the application, when the virtual makeup of eyebrow portion is carried out to the image to be processed, based on the key point in eyebrow portion region, determine the deformation zone to be processed, and carry out the zoom mapping processing to the deformation zone to be processed, reduce the eyebrow portion region, and then during the eyebrow portion that follow-up addition has the makeup effect, then can be fine to this eyebrow portion region cover, thereby the effect of making up that presents is better, furthermore, when carrying out the material and rendering, through establishing pixel mapping relation, make and can be independently parallel render between each pixel, thereby promote the efficiency that the image was rendered, the real-time that the video was made up has correspondingly been promoted.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, it is obvious that the drawings in the following description are only the embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;

fig. 2 is a schematic flowchart of an image rendering processing method according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of key points of an eyebrow area according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram illustrating a comparison of images before and after a scaling mapping process according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a deformation region to be processed of an elliptical model according to an embodiment of the present disclosure;

fig. 6 is a schematic flow chart illustrating a process of determining whether a pixel point a is an ellipse inner point according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a deformation region to be processed of a circular model according to an embodiment of the present disclosure;

fig. 8 is a schematic diagram of a pixel scaling mapping process according to an embodiment of the present disclosure;

fig. 9 is a schematic diagram of a constructed grid structure correspondence relationship provided in an embodiment of the present application;

fig. 10 is a grid structure diagram after an auxiliary key point is added according to an embodiment of the present application;

fig. 11 is a schematic flowchart of image rendering in a video stream according to an embodiment of the present disclosure;

fig. 12 is a schematic structural diagram of an image rendering processing apparatus according to an embodiment of the present application;

FIG. 13 is a schematic structural diagram of a computer device according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of a computing device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

For the convenience of understanding the technical solutions provided by the embodiments of the present application, some key terms used in the embodiments of the present application are explained first:

face key points: the key points mainly refer to key points of the eyebrows, and the outline features of the eyebrows can be described based on the key points of the eyebrows. In general, the detection of the face key points may be performed based on a face key point detection model, and the more common face key point models may be a face key point detection model based on 106 key points, a face key point detection model based on 256 key points, a face key point detection model based on 68 key points, and the like.

Scaling mapping: or referred to as liquefaction deformation, the method is to perform a certain deformation process on the eyebrow region and the peripheral region by using a liquefaction deformation processing method, so that the eyebrow region shows a shrinking effect in the face.

Standard human face: the method is an averaged face obtained by statistics based on a large amount of face data, and can represent the characteristic conditions of most faces.

The technical scheme of the embodiment of the application relates to an Artificial Intelligence (AI) technology, which is a theory, a method, a technology and an application system for simulating, extending and expanding human Intelligence by using a digital computer or a machine controlled by the digital computer, sensing the environment, acquiring knowledge and obtaining the best result by using the knowledge. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence base technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation interactive systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning, automatic driving, intelligent traffic and the like. With the development and progress of artificial intelligence, artificial intelligence is being researched and applied in a plurality of fields, for example, common fields such as smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, smart wearable devices, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, internet of vehicles, automatic driving, and smart transportation.

Machine Learning (ML) is the core of artificial intelligence, and is the fundamental approach to make computers have intelligence, and is applied throughout various fields of artificial intelligence; the core of machine learning is deep learning, which is a technology for realizing machine learning. Machine learning generally includes deep learning, reinforcement learning, migration learning, inductive learning, artificial Neural network, formal education learning, and the like, and deep learning includes Convolutional Neural Network (CNN), deep belief network, recurrent Neural network, auto-encoder, and generation countermeasure network, and the like.

Computer Vision (CV) is a comprehensive discipline integrating multiple disciplines such as Computer science, signal processing, physics, applied mathematics, statistics, neurophysiology and the like, and is also a challenging important research direction in the scientific field. Computer vision is a subject for studying how to make a machine "look", and more specifically, the subject is to use various imaging systems such as a camera and a computer to replace human visual organs, perform machine vision processing such as recognition, tracking and measurement on a target, and further process an acquired image into an image more suitable for human eyes to observe or to transmit to an instrument to detect through map forming processing.

Computer vision, a scientific discipline, attempts to make computers have the ability to observe and understand the world through visual organs like humans by studying related theories and technologies, and establishes an artificial intelligence system capable of acquiring information from images or multidimensional data. The computer vision technology generally includes image processing, image Recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content/behavior Recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, automatic driving, smart transportation, and other technologies, and besides, the computer vision technology also includes common biometric identification technologies such as face Recognition and fingerprint Recognition.

The Augmented Reality (AR) technology simulates and simulates physical information (such as visual information, sound information, taste information, tactile information and the like) which is difficult to experience in a certain time and space range of the real world originally, and then superimposes virtual information on the real world, so that two different kinds of information of the real world environment and the virtual world object are displayed to a user in the same picture or space, and the sensory experience beyond Reality is achieved.

The embodiment of the application mainly relates to technologies such as deep learning and computer vision in an artificial intelligence technology, namely a face detection model is obtained in a deep learning mode, the face detection model obtained through training is used for realizing the detection of key points of eyebrows in faces, so that subsequent eyebrow related processing is facilitated by combining a computer vision technology, and the specific process is explained through subsequent embodiments and is not repeated at first.

The following briefly introduces the design concept of the embodiments of the present application.

At present, in a video processing scene, such as a short video scene or a live video scene, the requirement of automatic face makeup often exists, eyebrows are an indispensable part in the makeup, and the good-looking eyebrows can effectively modify the face shape, enhance the three-dimensional sense of the face, and simultaneously enable the facial expression to be richer and more vivid, thereby directly influencing the makeup effect. Generally speaking, in actual life, the user often realizes through manual makeup eyebrow, and the eyebrow self condition of at first need judging confirms the eyebrow type, and then makes up through the eyebrow instrument, and this requires the user to have certain makeup experience, and the consumptive material that just takes time, is difficult to change different eyebrow types in the short time moreover.

In the related art, there is a scheme that eyebrows can be automatically dressed in an augmented reality manner, that is, eyebrows are identified by a detection or segmentation method to obtain an eyebrow region, a pure eyebrow template is subjected to affine transformation to obtain a deformed template, the deformed template is covered on the original eyebrows to obtain dressed eyebrows, or the original eyebrows are deducted and filled with colors of skin around the eyebrows, and then the textures of the eyebrow template are pasted.

However, in these schemes, an eyebrow template is often directly superimposed on an eyebrow region of a human face, and there may be a problem that the eyebrow template is not matched with an original eyebrow, so that the eyebrow template cannot completely cover the original eyebrow, and a makeup effect is not good. Based on this, the user can only select a suitable eyebrow template according to the eyebrow condition of the user, and can cover the original eyebrow, and when the user wants to try different eyebrow types, the related technical scheme still has certain limitation, and the better makeup effect can not be displayed. However, the way of picking up the original eyebrows still has a challenge in filling the surrounding skin, and if the filling is not proper, the surrounding of the eyebrows may show an abnormal display effect, such as uneven skin color, and further, a better makeup effect cannot be displayed.

Based on this, the embodiment of the application provides an image rendering processing method, in the method, when virtual makeup of an eyebrow part is performed on an image to be processed, a deformation area to be processed is determined based on key points of the eyebrow part area, scaling mapping processing is performed on the deformation area to be processed, the eyebrow part area is reduced, and then when the eyebrow part with a makeup effect is added subsequently, the eyebrow part area can be well covered, so that the presented makeup effect is better.

After introducing the design concept of the embodiment of the present application, some simple descriptions are provided below for application scenarios to which the technical solution of the embodiment of the present application can be applied, and it should be noted that the application scenarios described below are only used for describing the embodiment of the present application and are not limited. In a specific implementation process, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

The scheme provided by the embodiment of the application can be suitable for most scenes with image rendering processing, such as scenes with image beautifying requirements, such as a live broadcast platform and a content sharing platform, and can also be applied to daily scenes, such as simulated makeup tests of video calls, off-line special cabinets or on-line e-commerce platforms. As shown in fig. 1, an application scenario diagram provided in the embodiment of the present application may include a terminal device 101 and a background server 102 in the scenario.

The terminal device 101 may be, for example, a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, a smart television, a smart wearable device, and the like. The terminal device 101 may be installed with an application that can perform image beautification, such as a social application, a beauty application, or a photographing application.

The background server 102 may be a background server corresponding to an application installed on the terminal device 101. The background server 102 may be, for example, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform, but is not limited thereto.

The backend server 102 may include one or more processors 1021, memory 1022, and an I/O interface 1023 to interact with the terminal, among other things. In addition, the background server 102 may also configure a database 1024, and the database 1024 may be used to store model data, image rendering processing parameter data, and the like. The memory 1022 of the backend server 102 may further store program instructions of the image rendering processing method provided in the embodiment of the present application, and when the program instructions are executed by the processor 1021, the program instructions can be used to implement the steps of the image rendering processing method provided in the embodiment of the present application, so as to perform image rendering processing on an image to be processed, and obtain a target image with an eyebrow makeup effect.

In a possible implementation manner, a user provides an image to be processed in an application installed on the terminal device 101, for example, selects an image from an album, or shoots an image again, and the terminal device 101 may upload the image to be processed to the backend server 102, and the backend server 102 processes the image to be processed based on the image rendering processing method provided in the embodiment of the present application to obtain a target image with a virtual makeup effect, and then returns the target image to the terminal device 101 for display.

In another possible implementation manner, a user performs video acquisition by using an application installed on the terminal device 101, and transmits a video stream to the background server 102, and the background server 102 returns a video frame containing a human face to the terminal device 102 in real time after processing the video frame in the video stream based on the eyebrow template selected by the user, so that the terminal device 102 can present a video with a makeup effect in real time in the video acquisition process. In addition, in some scenarios, for example, in a live video scenario or a video call scenario, the backend server 102 may also send the processed video frame to the terminal devices 102 of other users, so that the terminal devices 102 of other users can also present videos with states.

In specific implementation, when the processing capability of the terminal device 101 allows, the process of the image rendering processing may also be performed on the terminal device 101, which is not limited in this embodiment of the application.

Terminal device 101 and backend server 102 may be in direct or indirect communication connection via one or more networks 103. The network 103 may be a wired network or a Wireless network, for example, the Wireless network may be a mobile cellular network, or may be a Wireless-Fidelity (WIFI) network, and of course, may also be other possible networks, which is not limited in this embodiment of the present application.

It should be noted that, in the embodiment of the present application, the number of the terminal devices 101 may be one, or may be multiple, that is, the number of the terminal devices 101 is not limited.

Of course, the method provided in the embodiment of the present application is not limited to be used in the application scenario shown in fig. 1, and may also be used in other possible application scenarios, and the embodiment of the present application is not limited. The functions that can be implemented by each device in the application scenario shown in fig. 1 will be described in the following method embodiments, and will not be described in detail herein.

Referring to fig. 2, a flowchart of an image rendering processing method provided in an embodiment of the present application is schematically illustrated, where the method may be executed by the background server 102 or the terminal device 101 in fig. 1, and a flow of the method is described as follows.

Step 201: the method comprises the steps of carrying out face key point identification on a first image to be processed containing a face, and identifying a plurality of first key points corresponding to a first actual eyebrow area of the face from the first image to be processed.

In the embodiment of the application, the first image to be processed may be a single image or a certain video frame in a video, when the first image to be processed is the single image, the image may be a shot image selected by a user from an album or an image shot again by the user, and when the first image to be processed is a certain video frame in the video, the video may also be a shot video selected by the user from the album or a video recorded in real time by the user.

In practical application, when a user requests to add a virtual makeup effect to a video, the user can perform face recognition on the video to determine whether an image to be processed containing a face exists, that is, an image to be processed which needs to be subjected to image rendering exists, and then perform subsequent rendering processing on the image to be processed which contains the face in the video.

In order to perform targeted processing on an eyebrow region subsequently, an actual area of the eyebrow needs to be identified from the image to be processed, where for differentiation, an original eyebrow region in the image to be processed is referred to as a first actual eyebrow region, and key points in the image to be processed are referred to as first key points, and then the area of the eyebrow, that is, a plurality of first key points needed need to be identified from the image to be processed, where an area surrounded by the first key points is the first actual eyebrow region.

Specifically, each first keypoint may be identified by a pre-trained face keypoint model, and any possible face keypoint model may be used, such as a 106-point-based face keypoint model, a 68-point-based face keypoint model, and a 256-point-based face keypoint model. Taking the face key point model based on 106 points as an example, see fig. 2, which shows a plurality of first key points of the first actual eyebrow region obtained by the face key point model based on 106 points, i.e. key points 1 to 19 shown in fig. 3, wherein key points 1 to 5 and 15 to 18 are key points of the left eyebrow, and key points 6 to 14 are key points of the right eyebrow.

In specific implementation, the face key point model needs to be trained in advance to be put into practical use. During training, a large number of training samples with face key point labels being performed in advance can be adopted to perform multiple-cycle iterative training on the constructed face key point model until the iteration times reach the set time condition or the accuracy of the face key point model reaches the set condition. The training sample can obtain labeled training data from an open-source training data set, and can label the acquired face image to form the training sample. The face key point model can adopt any possible deep neural network, and the embodiment of the application does not limit the depth neural network.

Step 202: and based on the plurality of first key points, determining a deformation area to be processed containing a first actual eyebrow area in the first image to be processed, and carrying out scaling mapping processing on each pixel point in the deformation area to be processed to obtain a second image to be processed containing a second actual eyebrow area after scaling.

In the embodiment of the application, in order to make the eyebrow portion of the rendered target image more realistic and more comfortable, the native eyebrow portion, that is, the first actual eyebrow portion region is liquefied and deformed, so that the first actual eyebrow portion region is zoomed, and a second to-be-processed image including the zoomed second actual eyebrow portion region is obtained.

Referring to fig. 4, a schematic diagram of a comparison between images before and after the scaling mapping process is shown, wherein before the scaling mapping process, a first actual brow region in the first generation processed image is consistent with a size of a native brow of a human face, and after the scaling mapping process, the first actual brow region is reduced in the human face, thereby obtaining a second image to be processed, as shown in fig. 4, in which the remaining portion except the eyebrow region is unchanged, the second actual eyebrow area is obviously reduced compared with the first actual eyebrow area, and the eyebrow in the second actual eyebrow area is obviously finer, so that after the eyebrow in the eyebrow template is added in subsequently, the eyebrow after being reduced can be easily covered, so that the eyebrow dressing effect after the eyebrow template is added is correspondingly improved.

In general, image processing is performed for one area, it is difficult to directly perform processing only for an eyebrow area without involving its vicinity, and in order to make the effect after processing more natural, it is necessary to perform processing for its vicinity together, and before performing scaling mapping processing, it is necessary to specify an area to be subjected to scaling mapping processing, that is, a deformation area to be processed.

In the embodiment of the present application, when determining the deformation region to be processed, the reference point of the deformation region to be processed may be determined in the first image to be processed based on the coordinate value of the first key point corresponding to the first actual eyebrow region in the constructed reference coordinate system, and further, the deformation region to be processed may be determined in the first image to be processed based on the set shape model with the reference point as the center.

The reference coordinate system is generally constructed based on the first image to be processed.

In one possible implementation, the reference coordinate system may be constructed by taking a center point of the first image to be processed as an original coordinate and taking horizontal and vertical directions as a horizontal axis and a vertical axis, respectively.

In another possible implementation, a reference coordinate system may be constructed by using one of the corner points of the first image to be processed as an original coordinate and using the horizontal and vertical directions as a horizontal axis and a vertical axis, respectively.

It should be noted that the above-mentioned to-be-processed deformation region is for a single-side eyebrow, that is, for a single face, two to-be-processed deformation regions are determined, one to-be-processed deformation region corresponds to one eyebrow, and then, the eyebrows on both sides are processed respectively.

In this embodiment of the application, a pixel point corresponding to an average of the coordinate values of each first key point may be determined as a reference point of the deformation region to be processed, that is, the reference point of the deformation region to be processed may be represented as follows:

wherein (x)₀，y₀) Is the coordinate value of the reference point, (x)_i，y_i) Is the coordinate value of the ith first key point, N is singleThe total number of first keypoints corresponding to the eyebrows.

Of course, the reference point may also be calculated in other possible manners, for example, after the shape fitting is performed on each first key point, a key point may be determined as the reference point, which is not limited in this embodiment of the application.

In the embodiment of the present application, the shape model may include, for example, an ellipse model, a circle model, a triangle model, or the like, and taking the ellipse model as an example, it means that the outer contour of the deformation region to be processed is an ellipse, and other shape models are similar.

The way of determining the deformation region to be processed is different based on different shape models, which are described below separately.

(1) Ellipse model

When the shape model is an ellipse model, the distances between the reference point and each first key point can be calculated one by one, the first distance with the largest value is determined from the obtained distances, and the semi-major axis of the ellipse is determined based on the first distance.

Taking the reference point as an example of the average value of the first key points, generally, the first key point at the tail of the eyebrow is the farthest distance from the reference point. Referring to fig. 5, the reference point is a pixel point 0, and the most distant point from the reference point 0 is generally a first key point 1, so that the semi-major axis of the ellipse can be determined based on the distance between the reference point 0 and the first key point, and a straight line formed by connecting the reference point 0 and the first key point is a straight line where the major axis of the ellipse is located.

In addition, a second distance with a larger value can be selected from the distances between the reference point and the two closest first key points, and the semi-minor axis of the ellipse is determined based on the second distance, so that the deformation area to be processed is obtained. Referring to fig. 5, for a schematic diagram of the constructed ellipse model, two first key points closest to the reference point 0 are 3 and 17, and then the distances between the reference point 0 and the first

key points

3 and 17 can be calculated respectively, and a larger value is selected to be used as the reference distance of the semi-minor axis.

In one possible embodiment, the first distance may be taken as the semi-major axis of the ellipse and the second distance may be determined as the semi-minor axis. Further based on the fiducial points, semi-major axis and semi-minor axis then the following ellipse equation can be determined:

wherein d is₁Is the length of the semi-major axis of the ellipse, i.e. the first distance, d₂Is the semi-minor axis length of the ellipse, i.e., the second distance described above.

In another possible embodiment, in order to avoid the situation that the ellipse cannot enclose all the eyebrow region, a certain magnification factor may be set, that is, after the first distance and the second distance are magnified, the magnified first distance and the magnified second distance are respectively used as the semi-major axis and the semi-minor axis to construct the ellipse region, which is specifically illustrated in fig. 5 by taking this as an example. Further, the ellipse equation can be expressed as follows:

alpha and beta are respectively the magnification times corresponding to the semi-major axis and the semi-minor axis, and the values of alpha and beta can be the same or different, can be set according to empirical values, and can also be adjusted according to the actual eyebrow condition.

In the embodiment of the present application, after the ellipse model is constructed, the ellipse region may be used as the deformation region to be processed to perform scaling mapping processing, that is, the region enclosed by the ellipse shown in fig. 5 is used as the deformation region to be processed to perform scaling mapping processing.

Specifically, in order to subsequently know which pixel points are subjected to scaling mapping processing, it is necessary to further determine which pixel points are located in the elliptical region based on the elliptical region constructed as described above. Since the determination process of each pixel is similar, only pixel a is taken as an example to illustrate here. Fig. 6 is a schematic flow chart showing the process of determining whether the pixel point a is an ellipse inner point.

S1: and determining an included angle value between a connecting line between the reference point and the first key point corresponding to the first distance and a set coordinate axis in the reference coordinate system.

Since the coordinates of each reference point, pixel point, or first key point are coordinate values in the reference coordinate system, the coordinates need to be converted into a coordinate system in which the ellipse is located for distance calculation.

S2: and taking the reference point as a center, rotating the pixel point A by the included angle value, and determining the distance between the position of the rotated pixel point A and the reference point.

Referring to fig. 5, a certain included angle Θ exists between a horizontal axis and a line connecting the reference point and the first key point 1, so that when a coordinate system is converted, the pixel point needs to be rotated, and if a coordinate value of the pixel point a in the reference coordinate system is (x)_a，y_a) Then, the coordinate value after the rotation angle theta is (x)_a′，y_a') the distance between the pixel point a and the reference point 0 can be defined as (here, the above first embodiment is taken as an example):

wherein r is the distance from the pixel point A to the center of the ellipse.

S3: and determining whether the distance between the position of the rotated pixel point A and the reference point is smaller than a set distance threshold value.

S4: if the determination result in the step S3 is negative, it is determined that the pixel point a is a pixel point not belonging to the deformation region to be processed.

S5: if the determination result in the step S3 is yes, the pixel point a is determined to be a pixel point located in the deformation region to be processed.

Specifically, for the above distance definition, the set distance threshold may be set to 1, that is, when r is less than 1, the pixel point a belongs to an ellipse inner point, that is, a pixel point in the deformation region to be processed, which needs to be subjected to the scaling mapping processing, and when r is greater than or equal to 1, the pixel point a belongs to an ellipse outer point or a point on an ellipse line, that is, a pixel point in the deformation region to be processed, which does not belong to the scaling mapping processing.

(2) Circular model

When the shape model is a circular model, distances between the reference point and each first key point can be calculated one by one, a first distance with the largest value is determined from the obtained distances, and the circular radius is determined based on the first distance.

Also taking the reference point as the average value of the first key points, generally speaking, the first key point at the end of the eyebrow is the farthest distance from the reference point. Referring to fig. 7, the reference point is a pixel point 0, and the most distant from the reference point 0 is generally the first key point 1, so that the circle radius can be determined based on the distance between the reference point 0 and the first key point.

In one possible embodiment, the first distance may be taken as a circle radius, which is shown in detail in fig. 7, and based on the reference point and the circle radius, the following circle equation may be determined:

wherein d is₁Is a circular radius, i.e. the first distance mentioned above.

In another possible embodiment, a certain magnification factor may also be set, that is, after the first distance is magnified, the magnified first distance is used as a radius to construct the circular area. Further, the ellipse equation can be expressed as follows:

similarly, the manner of determining whether each pixel point is a point in the circular region is similar to that of the above-described ellipse model, and therefore, the details are not repeated herein.

It should be noted that any other possible shape models can also be adopted in the embodiments of the present application, and this is not limited thereto.

In the embodiment of the present application, for each pixel point in the determined deformation region to be processed, scaling mapping processing is performed in the following manner, where a first key point 1 in the deformation region to be processed is taken as an example to show.

For the first keypoint 1, the scaled coordinate value of the first keypoint 1 may be determined based on the coordinate value of the first keypoint 1 and the set scaling mapping relationship, and then, the pixel point at the position where the scaled coordinate value is located is updated according to the pixel characteristic value of the first keypoint 1, so as to obtain the second image to be processed including the scaled second actual eyebrow region as shown in fig. 4.

Fig. 8 is a schematic diagram of the pixel scaling and mapping process. It can be seen that, after the original eyebrow is reduced, the first key point 1 on the original eyebrow is translated to the position of the pixel point 1', which is to substantially replace the pixel value at the position of the pixel point 1' with the pixel feature value of the first key point 1.

Specifically, in the set scaling mapping relationship adopted in the embodiment of the present application, the scaling ratio is in negative correlation with the distance between the reference points of the deformation region to be processed, that is, the deformation of the pixel points farther from the reference points of the deformation region to be processed is more obvious, and the deformation of the points closer to the edge of the deformation region to be processed is smaller until the edge of the deformation region to be processed is not deformed, so the scaling mapping relationship satisfying the deformation relationship is applicable.

In one possible implementation, the following scaling mapping relationship may be employed:

f(r)＝(1+(r-1)²*ε)

where f (r) is the scaling mapping expression, ε is the degree to which the eyebrows are scaled, and the range of values is [0, 1 ].

Therefore, for any pixel point (x, y) in the deformation region to be processed, the mapped pixel point position is (x ', y') as follows:

in general, since the eyebrows are elongated along the x-axis, the eyebrows may be mapped by using asymmetric scaling, for example, different scaling of the left and right parts of the eyebrows, different scaling of the upper and lower parts, or the like, in addition to the above-described scaling mapping relationship.

In the embodiment of the application, because the number of the pixels before the reduction is greater than the number of the pixels after the reduction, some pixels cannot be mapped to the pixels after the reduction, and the pixels are discarded, or a plurality of pixels are mapped to the same pixel after the reduction, so that the pixel after the reduction can select the characteristic mean value or the characteristic maximum value of the plurality of pixels.

Step 203: and acquiring a target eyebrow template containing a virtual makeup effect, and establishing a one-to-one corresponding pixel mapping relation between the first actual eyebrow area and the simulated eyebrow area based on the first key points and the second key points corresponding to the simulated eyebrow area in the eyebrow template.

In one possible implementation, the target eyebrow template may be obtained by triggering acquisition through a selection operation of a user. For example, the user may select a target eyebrow template from the provided eyebrow templates before shooting the video or image, or may select a corresponding target eyebrow template for the image after shooting the video or image.

In another possible implementation, after face recognition is performed on the image to be processed provided by the user, the most appropriate target eyebrow module is matched for the face.

In order to render the second image to be processed by the eyebrow material in real time, the pixel mapping relation corresponding to the first actual eyebrow area and the simulated eyebrow area one by one can be established, so that rendering processing of all pixel points can be independently and parallelly performed, time complexity is greatly reduced, and processing speed is improved.

Specifically, when the pixel mapping relationship is constructed, the first actual eyebrow region and the simulated eyebrow region may be respectively subjected to gridding in advance to obtain the corresponding first grid structure and second grid structure, and then the pixel mapping relationship is established based on the grid correspondence relationship between the first grid structure and the second grid structure.

In the first grid structure, the first key point is a vertex of a grid in the first grid structure, and one grid corresponds to one sub-region of the first actual eyebrow area; similarly, in the second mesh structure, the plurality of second key points in the simulated eyebrow region are vertices of meshes in the second mesh structure, and one mesh corresponds to one sub-region of the simulated eyebrow region, and the meshes in the first mesh structure correspond to the meshes of the second mesh structure one to one.

In a possible implementation manner, the mesh may be constructed by a mesh construction method based on a triangular mesh, and the eyebrow region is triangulated.

Specifically, the mesh construction method based on the triangular mesh can be based on a given key point set, the key points are used as end points to form closed line segments, a triangular mesh structure is constructed by the closed line segments, and the triangular mesh structure meets the following conditions:

(1) except for the end points, an edge in a triangular mesh structure does not contain any point in the set of points.

(2) There are no intersecting edges.

(3) All faces in the triangular mesh structure are triangular faces, and the set of all triangular faces is a convex hull of the set of keypoints.

Among them, the triangulation method that is more commonly used is the Delaunay triangulation (Delaunay triangulation algorithm), which is a special triangulation. The Delaunay edge in the Delaunay algorithm satisfies the property of a hollow circle, that is, a closed line segment e formed by two

key points

1 and 2 in the key point set satisfies that a circle passes through two

points

1 and 2, and the closed line segment e does not include any other point in the point key point set in the circle, and satisfies the property of the hollow circle, and is called a Delaunay edge, so that if a triangulation of the key point set only includes the Delaunay edge, the triangulation is called a Delaunay triangulation.

Furthermore, the triangular mesh structure obtained by the Delaunay triangulation satisfies the following conditions:

any triangulation T in the triangulated mesh structure, satisfying that T is a Delaunay triangulation of the set of keypoints, is currently only if the inside of the circumscribed circle of each triangle in T does not contain any of the points in the set of keypoints.

In the embodiment of the application, no matter the key points of the actual eyebrow area in the image to be processed or the key points of the simulated eyebrow area in the eyebrow template are consistent in number, the only difference is that the distances between the key points are different due to the different shapes of the eyebrows, but the relative position relations of the key points are also consistent, so that the connection relation of the grid structure can be constructed in advance based on the standard human face, namely, which key points in the finally constructed grid structure are connected to form a subdivision surface, and therefore in practical application, the connection relation can be directly quoted, meshing processing can be rapidly carried out, and the grid structure can be constructed.

Fig. 9 is a schematic diagram of the constructed grid structure correspondence relationship. In the first actual brow area, each first keypoint constitutes a first grid structure, the

first keypoints

1, 2 and 18 shown in fig. 9 constitute a grid, the grid corresponds to a sub-area of the first actual brow area, and the rest is analogized in sequence. Similarly, in the simulated eyebrow area, each second keypoint also constitutes a second grid structure, and the connection relationship in the second grid structure is consistent with the first grid structure, as shown in fig. 9, the

second keypoints

1, 2, and 18 constitute a grid corresponding to a sub-area of the simulated eyebrow area, and the grid constituted by the

second keypoints

1, 2, and 18 corresponds to the grid constituted by the

second keypoints

1, 2, and 18 in the first grid structure, the dashed arrows shown in fig. 9 indicate the correspondence relationship, and the rest are analogized in sequence.

Further, based on the correspondence between each mesh in the first actual eyebrow region and each mesh in the simulated eyebrow region, a pixel mapping relationship is constructed for each corresponding mesh, for example, a mesh composed of the first

key points

1, 2, and 18 and a mesh composed of the second

key points

1, 2, and 18 shown in fig. 9, and a pixel mapping relationship between the two meshes is established.

In the embodiment of the present application, as shown in fig. 9, in consideration that a mesh constructed by key points may not cover all areas of an eyebrow, auxiliary key points may be added, so that when performing the meshing process, the meshing process may be performed based on a set meshing process manner with each key point and at least one set auxiliary key point as vertices, and a corresponding mesh structure is obtained. Referring to fig. 10, in order to obtain a grid structure diagram after adding the auxiliary key points, where points a to K are the auxiliary key points, it can be seen that after adding the auxiliary key points, all areas of the eyebrow area can be well included, so that the make-up effect is more consistent with the real effect.

The at least one auxiliary key point is a pixel point located outside the first actual eyebrow region, the gridding processing mode is determined based on a standard eyebrow region in a standard face image, the standard face image is a face image in a front-view state obtained by averaging various types of face templates, and the set gridding processing mode is such as the previously obtained grid connection relation.

Step 204: based on the pixel mapping relation, the rendering materials of all the pixel points are extracted from the simulated eyebrow area, and in the second image to be processed, fusion rendering processing is carried out on the corresponding pixel points respectively to obtain the target image.

In the embodiment of the application, a pixel mapping relation is established by a gridding-based mapping method, and then, a texture mapping technology of computational graphics can be utilized to sample a material from an eyebrow template, so that the texture of the material is mapped to a second image to be processed, and a target image is obtained.

For example, a texture mapping method based on a triangular mesh may be adopted to triangulate a simulated eyebrow region in an eyebrow template, that is, key points of eyebrows are used as vertices of the mesh, a Delaunay algorithm is used to obtain a triangular mesh structure of the eyebrow region, and similarly, for a first actual eyebrow region in an image to be processed, such a triangular mesh structure may also be constructed, so that a topological structure of the triangular mesh is used to sample from the eyebrow template, thereby obtaining a texture mapping of a second image to be processed after the eyebrows are reduced by the eyebrow template.

The image rendering processing process in the embodiment of the present application may be applied to various image processing scenes, for example, image rendering in a video stream, and the following description is given by taking image rendering in a video stream as an example. Referring to fig. 11, a schematic flowchart of rendering an image in a video stream is shown.

Step 1101: and acquiring a video stream to be processed.

Specifically, the video stream to be processed may be selected by the user from a local album, or may be shot by the user in real time.

Step 1102: it is determined whether the current video frame contains a human face.

Step 1103: if the result of step 1102 is yes, the current video frame is determined as the image to be processed.

Step 1104: and performing image rendering processing on the image to be processed to obtain a corresponding target image.

Step 1105: and obtaining a target video stream containing the virtual makeup effect based on the target image corresponding to each image to be processed.

Step 1106: if the result of step 1102 is negative, skipping to the next video frame for further processing.

In the embodiment of the present application, the rendering processing on each to-be-processed image may be performed sequentially according to a queue, or may be performed in parallel with the rendering processing on a plurality of to-be-processed images.

In a possible implementation, the processing of the video stream may be performed locally by the terminal device.

In another possible implementation, in the processing procedure of the video stream, the terminal device may upload the video stream to a background server, and the background server provides a background processing service. For example, after the user selects the virtual makeup effect on the shooting application of the terminal device, the shooting application is used for starting to shoot the video, correspondingly, the video stream is uploaded to the background server, and the background server processes each video frame by using the processing flow and returns the target video stream to the terminal device, so that the terminal device displays the target video stream in real time, and the user can view the target video stream added with the virtual makeup effect.

To sum up, this application embodiment adopts augmented reality's method, through carrying out liquefaction deformation to the eyebrow region, carries out local zoom to the eyebrow, then rethread augmented reality mode, different eyebrows of laminating obtain the eyebrow type diversified, the effect of making up naturally, wherein eyebrow portion reduces the scheme and makes the eyebrow template can not take place obvious mixed stack with native eyebrow portion, make up the effect of making up of eyebrow portion more laminate nature, and, through the texture laminating that utilizes the graphics, carry out the texture sampling to the eyebrow portion template, make up in real time for the eyebrow.

Referring to fig. 12, based on the same inventive concept, an embodiment of the present application further provides an image rendering processing apparatus 120, including:

a face recognition unit 1201, configured to perform face keypoint recognition on a first to-be-processed image including a face, and recognize a plurality of first keypoints corresponding to a first actual eyebrow region of the face from the first to-be-processed image;

a scaling mapping unit 1202, configured to determine, based on the plurality of first key points, a to-be-processed deformation region including a first actual eyebrow region in the first to-be-processed image, and perform scaling mapping processing on each pixel point in the to-be-processed deformation region to obtain a second to-be-processed image including a scaled second actual eyebrow region;

a mesh constructing unit 1203, configured to obtain a target eyebrow template including a virtual makeup effect, and establish a one-to-one corresponding pixel mapping relationship between a first actual eyebrow region and a simulated eyebrow region based on a plurality of first key points and a plurality of second key points corresponding to the simulated eyebrow region in the eyebrow template;

and the texture attaching unit 1204 is configured to extract rendering materials of each pixel point from the simulated eyebrow region based on the pixel mapping relationship, and perform fusion rendering processing on the corresponding pixel points in the second image to be processed, so as to obtain a target image.

Optionally, the scaling mapping unit 1202 is specifically configured to:

determining a reference point of a deformation region to be processed in a first image to be processed based on coordinate values of a plurality of first key points in the constructed reference coordinate system;

based on the set shape model, a deformation region to be processed is determined in the first image to be processed centering on the reference point.

Optionally, the scaling mapping unit 1202 is specifically configured to:

and determining pixel points corresponding to the mean value of the coordinate values corresponding to the first key points as reference points.

Optionally, the scaling mapping unit 1202 is specifically configured to:

determining the distance between the reference point and each first key point, determining a first distance with the largest value from the obtained distances, and determining the semi-major axis based on the first distance;

and determining an elliptical area surrounded by the semi-major axis and the semi-minor axis by taking the reference point as the center as a deformation area to be processed.

Optionally, the scaling mapping unit 1202 is specifically configured to:

determining an included angle value between a connecting line between the reference point and a first key point corresponding to the first distance and a set coordinate axis in the reference coordinate system;

for each pixel point in the image to be processed, the following operations are respectively executed:

for one pixel point, taking the reference point as a center, rotating the included angle value of the pixel point, and determining the distance between the position of the rotated pixel point and the reference point;

and if the distance between the position of the rotated pixel point and the reference point is smaller than a set distance threshold, determining that one pixel point is the pixel point in the deformation area to be processed.

Optionally, the scaling mapping unit 1202 is specifically configured to:

and determining a circular area which takes the reference point as the center and takes the maximum distance as the radius as a deformation area to be processed.

Optionally, the scaling mapping unit 1202 is specifically configured to:

and respectively executing the following operations for each pixel point in the deformation region to be processed to obtain a second image to be processed:

and updating the pixel point of the position of the scaled coordinate value according to the pixel characteristic value of one pixel point.

Optionally, the grid constructing unit 1203 is specifically configured to:

based on the plurality of first key points, carrying out gridding processing on the first actual eyebrow area to obtain a first grid structure; wherein, the first key points are the vertexes of the grids in the first grid structure, and one grid corresponds to a sub-area of the first actual eyebrow area; and the number of the first and second groups,

based on the plurality of second key points, carrying out gridding processing on the simulated eyebrow area to obtain a second grid structure; the plurality of second key points are the vertexes of grids in the second grid structure, one grid corresponds to one sub-region of the simulated eyebrow region, and the grids in the first grid structure correspond to the grids in the second grid structure one to one;

and establishing a pixel mapping relation based on the grid corresponding relation between the first grid structure and the second grid structure.

Optionally, the grid constructing unit 1203 is specifically configured to:

Optionally, the apparatus further comprises a video stream processing unit 1205 for:

acquiring a video stream to be processed, and selecting an image containing a human face from the video stream to be processed as an image to be processed; and obtaining a target video stream containing the virtual makeup effect based on the target image corresponding to each image to be processed.

The apparatus may be configured to execute the methods shown in the embodiments shown in fig. 2 to 11, and therefore, for functions and the like that can be realized by each functional module of the apparatus, reference may be made to the description of the embodiments shown in fig. 2 to 11, which is not repeated here. Among them, the video stream processing unit 1205 is an optional functional block, and is thus shown by a dotted line in fig. 12.

Based on the same technical concept as the method embodiment, the embodiment of the present application further provides a computer device 130, which may include a memory 1301 and a processor 1302.

The memory 1301 is used for storing computer programs executed by the processor 1302. The memory 1301 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to use of the computer device, and the like. The processor 1302 may be a Central Processing Unit (CPU), a digital processing unit, or the like. The specific connection medium between the memory 1301 and the processor 1302 is not limited in this embodiment. In the embodiment of the present application, the memory 1301 and the processor 1302 are connected through a bus 1303 in fig. 13, the bus 1303 is shown by a thick line in fig. 13, and the connection manner between other components is merely an illustrative description and is not limited thereto. The bus 1303 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 13, but this is not intended to represent only one bus or type of bus.

The memory 1301 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 1301 may also be a non-volatile memory (non-volatile memory) such as, but not limited to, a read-only memory (rom), a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD), or the memory 1301 may be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Memory 1301 may be a combination of the above.

A processor 1302, configured to execute the method performed by the apparatus in the embodiments shown in fig. 2 to fig. 11 when calling the computer program stored in the memory 1301.

The computing device 140 according to this embodiment of the present application is described below with reference to fig. 14. The computing device 140 of fig. 14 is only one example and should not impose any limitations on the functionality or scope of use of embodiments of the present application.

As shown in fig. 14, computing device 140 is embodied in the form of a general purpose computing device. Components of computing device 140 may include, but are not limited to: the at least one processing unit 1401, the at least one memory unit 1402, and a bus 1403 connecting the different system components (including the memory unit 1402 and the processing unit 1401).

Bus 1403 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

The storage unit 1402 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)14021 and/or cache storage unit 14022, and may further include Read Only Memory (ROM) 14023.

Storage unit 1402 may also include a program/utility 14025 having a set (at least one) of program modules 14024, such program modules 14024 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The computing device 140 may also communicate with one or more external devices 1404 (e.g., keyboard, pointing device, etc.), and may also communicate with one or more other computing devices to enable a user to communicate with, and/or communicate with, any device (e.g., router, modem, etc.) that enables the computing device 140 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 1405. Also, computing device 140 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through network adapter 1406. As shown, the network adapter 1406 communicates with other modules for the computing device 140 over a bus 1403. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computing device 140, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In some possible embodiments, various aspects of the methods provided by the present application may also be implemented in the form of a program product including program code for causing a computer device to perform the steps of the methods according to various exemplary embodiments of the present application described above in this specification when the program product is run on the computer device, for example, the computer device may perform the methods performed by the devices in the embodiments shown in fig. 2-11.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. An image rendering processing method, characterized in that the method comprises:

2. The method of claim 1, wherein determining a deformed region to be processed containing the first actual brow region in the first image to be processed based on the plurality of first keypoints comprises:

3. The method of claim 2, wherein determining the reference point of the deformation region to be processed in the first image to be processed based on the coordinate values of the plurality of first keypoints in the constructed reference coordinate system comprises:

4. The method of claim 2, wherein the shape model is an ellipse model, and determining the deformation region to be processed in the first image to be processed based on the set shape model centering on the reference point comprises:

5. The method as claimed in claim 4, wherein before performing scaling mapping processing on each pixel point in the deformation region to be processed to obtain a second image to be processed including a scaled second actual eyebrow region, the method further comprises:

6. The method of claim 2, wherein the shape model is a circular model, and determining the deformation region to be processed in the first image to be processed based on the set shape model centering on the reference point comprises:

7. The method according to any one of claims 1 to 6, wherein performing scaling mapping processing on each pixel point in the deformation region to be processed to obtain a second image to be processed including a scaled second actual eyebrow region comprises:

8. The method according to any one of claims 1 to 6, wherein establishing a one-to-one correspondence pixel mapping relationship between the first actual brow region and the simulated brow region based on the plurality of first keypoints and a plurality of second keypoints corresponding to the simulated brow region in the brow template comprises:

9. The method of claim 8, wherein gridding the first actual brow region based on the plurality of first keypoints to obtain a first grid structure comprises:

10. The method according to any one of claims 1 to 6, wherein before performing face keypoint recognition on an image to be processed containing a face, and recognizing a plurality of first keypoints corresponding to an actual eyebrow region of the face from the image to be processed, the method further comprises:

acquiring a video stream to be processed, and selecting an image containing a human face from the video stream to be processed as the image to be processed;

based on the feature mapping relationship, extracting rendering materials of each pixel point from the eyebrow template, and performing fusion rendering processing on corresponding pixel points in the image to be processed after the eyebrow is reduced to obtain a target image, the method further includes:

and obtaining a target video stream containing the virtual makeup effect based on the target image corresponding to each image to be processed.

11. An image rendering processing apparatus, characterized in that the apparatus comprises:

12. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor,

the processor, when executing the computer program, realizes the steps of the method of any one of claims 1 to 10.

13. A computer storage medium having computer program instructions stored thereon, wherein,

the computer program instructions, when executed by a processor, implement the steps of the method of any one of claims 1 to 10.