CN109816791A

CN109816791A - Method and apparatus for generating information

Info

Publication number: CN109816791A
Application number: CN201910100632.8A
Authority: CN
Inventors: 郭冠军
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Douyin Vision Co Ltd; Douyin Vision Beijing Co Ltd
Priority date: 2019-01-31
Filing date: 2019-01-31
Publication date: 2019-05-28
Anticipated expiration: 2039-01-31
Also published as: CN109816791B; WO2020155908A1

Abstract

Embodiment of the disclosure discloses the method and apparatus for generating information.One specific embodiment of this method includes: to obtain to carry out shooting Zuoren face image obtained and right facial image to target face；For the facial image in Zuoren face image and right facial image, following steps are executed: facial image input mapping graph trained in advance being generated into model, obtains mapping graph corresponding to the facial image；For the point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph determines the three-dimensional coordinate of the corresponding face key point of the point；Based on the three-dimensional coordinate of face key point identified, in the facial image, three-dimensional grid corresponding to the facial image is generated；Based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, result three-dimensional grid corresponding to target face is generated.The embodiment helps to improve the accuracy of three-dimensional facial reconstruction.

Description

Method and apparatus for generating information

Technical field

Embodiment of the disclosure is related to field of computer technology, more particularly, to generates the method and apparatus of information.

Background technique

With popularizing for mobile video application, various face special effective functions are also widely used.Three-dimensional face weight The technology as a kind of effective face statement is built, is had wide practical use.

Three-dimensional facial reconstruction is the three dimensional network that face is directly returned by the Pixel Information of given two-dimension human face image The process of lattice information (3D mesh).The two-dimension human face image for being commonly used for three-dimensional facial reconstruction is to utilize the electricity for including camera Sub- equipment (such as mobile phone, camera etc.), obtains image from some angle shot face.

Summary of the invention

Embodiment of the disclosure proposes the method and apparatus for generating information.

In a first aspect, embodiment of the disclosure provides a kind of method for generating information, this method comprises: acquisition pair Target face carries out shooting Zuoren face image obtained and right facial image, wherein Zuoren face image and right facial image are Binocular vision image；For the facial image in Zuoren face image and right facial image, following steps are executed: by the facial image Input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image, wherein point in mapping graph with Face key point in the facial image is corresponding；For the point in mapping graph, coordinate based on this in mapping graph and should The pixel value of point, determines the three-dimensional coordinate of the corresponding face key point of the point；Based on face identified, in the facial image The three-dimensional coordinate of key point generates three-dimensional grid corresponding to the facial image；Based on three dimensional network corresponding to Zuoren face image Three-dimensional grid corresponding to lattice and right facial image generates result three-dimensional grid corresponding to target face.

In some embodiments, the pixel value of the coordinate based on this in mapping graph and the point determines that the point is corresponding The three-dimensional coordinate of face key point, comprising: the pixel value based on the point determines the depth value of the corresponding face key point of the point； Coordinate of the corresponding face key point of the point in the facial image is determined based on coordinate of this in mapping graph；Based on this point Coordinate of the depth value of the corresponding face key point face key point corresponding with the point in the facial image, determines the point pair The three-dimensional coordinate for the face key point answered.

In some embodiments, the pixel value based on the point determines the depth value of the corresponding face key point of the point, packet It includes: in response to determining that the pixel value of the point is more than or equal to preset threshold, the pixel value of the point being determined as the corresponding face of point The depth value of key point.

In some embodiments, the pixel value based on the point determines the depth value of the corresponding face key point of the point, also wraps It includes: in response to determining that the pixel value of the point is less than preset threshold, preset threshold being determined as the corresponding face key point of the point Depth value.

In some embodiments, based on three-dimensional corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image Grid generates result three-dimensional grid corresponding to target face, comprising: crosses three-dimensional grid corresponding to Zuoren face image respectively The center line of three-dimensional grid corresponding to center line and right facial image, establishes datum level, wherein datum level runs through along center line Three-dimensional grid is divided into two parts by three-dimensional grid；It extracts in three-dimensional grid corresponding to Zuoren face image, it is left to be located at datum level The three-dimensional grid of side is as left three-dimensional grid, and extracts in three-dimensional grid corresponding to right facial image, and it is right to be located at datum level The three-dimensional grid of side is as right three-dimensional grid；Extracted left three-dimensional grid and right three-dimensional grid are spliced, target is generated Result three-dimensional grid corresponding to face.

In some embodiments, mapping graph generates model and is obtained by following steps training: training sample set is obtained, In, training sample includes the coordinate and depth value of sample facial image, face key point in sample facial image；For training Training sample in sample set determines the face in the training sample based on the coordinate of the face key point in the training sample Mapping position of the key point in mapping graph to be built, and the depth value based on the face key point in the training sample, The pixel value for determining corresponding mapping position in mapping graph to be built, utilizes the picture of identified mapping position and mapping position Element value, constructs mapping graph corresponding with the sample facial image in the training sample；Using machine learning method, by training sample The sample facial image of the training sample of concentration as input, using mapping graph corresponding to the sample facial image inputted as Desired output, training obtain mapping graph and generate model.

In some embodiments, the training sample that training sample is concentrated is generated by following steps: utilizing depth map acquisition The face depth map of device collecting sample face, and obtain facial image corresponding to face depth map；To face depth map Corresponding facial image carries out face critical point detection, to determine that the face in facial image corresponding to face depth map closes The coordinate of key point；By facial image corresponding to face depth map, the coordinate of identified face key point, based on face depth The depth value for scheming determining face key point summarizes for training sample.

Second aspect, embodiment of the disclosure provide a kind of for generating the device of information, which includes: that image obtains Unit is taken, acquisition is configured to and target face is carried out to shoot Zuoren face image obtained and right facial image, wherein Zuoren Face image and right facial image are binocular vision image；First generation unit is configured to for Zuoren face image and right face Facial image in image executes following steps: facial image input mapping graph trained in advance being generated model, is somebody's turn to do Mapping graph corresponding to facial image, wherein the point in mapping graph is corresponding with the face key point in the facial image；For Point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph, determines the corresponding face key point of the point Three-dimensional coordinate；Based on the three-dimensional coordinate of face key point identified, in the facial image, it is right to generate facial image institute The three-dimensional grid answered；Second generation unit is configured to based on three-dimensional grid corresponding to Zuoren face image and right facial image Corresponding three-dimensional grid generates result three-dimensional grid corresponding to target face.

In some embodiments, the first generation unit is further configured to: the pixel value based on the point, determines the point pair The depth value for the face key point answered；Determine the corresponding face key point of the point in the people based on coordinate of this in mapping graph Coordinate in face image；Depth value face key point corresponding with the point based on the corresponding face key point of the point is in the face Coordinate in image determines the three-dimensional coordinate of the corresponding face key point of the point.

In some embodiments, the first generation unit is further configured to: in response to determining that the pixel value of the point is greater than Equal to preset threshold, the pixel value of the point is determined as to the depth value of the corresponding face key point of the point.

In some embodiments, the first generation unit is further configured to: in response to determining that the pixel value of the point is less than Preset threshold is determined as the depth value of the corresponding face key point of the point by preset threshold.

In some embodiments, the second generation unit includes: to establish module, and it is right to be configured to cross Zuoren face image institute respectively The center line of three-dimensional grid corresponding to the center line for the three-dimensional grid answered and right facial image, establishes datum level, wherein benchmark Three-dimensional grid is run through along center line in face, and three-dimensional grid is divided into two parts；Extraction module is configured to extract Zuoren face image In corresponding three-dimensional grid, the three-dimensional grid on the left of datum level is as left three-dimensional grid, and extracts right facial image In corresponding three-dimensional grid, the three-dimensional grid on the right side of datum level is as right three-dimensional grid；Splicing module, is configured to pair Extracted left three-dimensional grid and right three-dimensional grid are spliced, and result three-dimensional grid corresponding to target face is generated.

The third aspect, embodiment of the disclosure provide a kind of electronic equipment, comprising: one or more processors；Storage Device is stored thereon with one or more programs, when one or more programs are executed by one or more processors, so that one Or the method that multiple processors realize any embodiment in the above-mentioned method for generating information.

Fourth aspect, embodiment of the disclosure provide a kind of computer-readable medium, are stored thereon with computer program, The program realizes any embodiment in the above-mentioned method for generating information method when being executed by processor.

The method and apparatus for generating information that embodiment of the disclosure provides clap target face by obtaining Take the photograph Zuoren face image obtained and right facial image, wherein Zuoren face image and right facial image are binocular vision image, and Afterwards for the facial image in Zuoren face image and right facial image, following steps are executed: the facial image is inputted into instruction in advance Experienced mapping graph generates model, obtains mapping graph corresponding to the facial image, wherein point and the facial image in mapping graph In face key point it is corresponding；For the point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph, Determine the three-dimensional coordinate of the corresponding face key point of the point；Based on the three of face key point identified, in the facial image Coordinate is tieed up, three-dimensional grid corresponding to the facial image is generated, finally based on three-dimensional grid corresponding to Zuoren face image and the right side Three-dimensional grid corresponding to facial image generates result three-dimensional grid corresponding to target face.It is appreciated that due to blocking and The reasons such as shooting angle, Zuoren face image and right facial image can recorde the face characteristic of different angle, so here, utilizing Three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, can be generated more accurate, mesh Result three-dimensional grid corresponding to face is marked, the accuracy of three-dimensional facial reconstruction is helped to improve.

Detailed description of the invention

By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the disclosure is other Feature, objects and advantages will become more apparent upon:

Fig. 1 is that one embodiment of the disclosure can be applied to exemplary system architecture figure therein；

Fig. 2 is the flow chart according to one embodiment of the method for generating information of the disclosure；

Fig. 3 is according to an embodiment of the present disclosure for generating the schematic diagram of an application scenarios of the method for information；

Fig. 4 is the flow chart according to another embodiment of the method for generating information of the disclosure；

Fig. 5 is the structural schematic diagram according to one embodiment of the device for generating information of the disclosure；

Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the electronic equipment of embodiment of the disclosure.

Specific embodiment

The disclosure is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to Convenient for description, part relevant to related invention is illustrated only in attached drawing.

It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure can phase Mutually combination.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

Fig. 1 is shown can be using the method for generating information of the disclosure or the implementation of the device for generating information The exemplary system architecture 100 of example.

As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105. Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with Including various connection types, such as wired, wireless communication link or fiber optic cables etc..

User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out Send message etc..Various telecommunication customer end applications can be installed, such as image processing class is soft on terminal device 101,102,103 Part, video playback class software, web browser applications, searching class application, instant messaging tools, social platform software etc..

Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard When part, it can be various electronic equipments, including but not limited to smart phone, tablet computer, E-book reader, MP3 player (Moving Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio level 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard audio level 4) player, pocket computer on knee and desktop computer etc..It, can be with when terminal device 101,102,103 is software It is mounted in above-mentioned cited electronic equipment.Multiple softwares or software module may be implemented into (such as providing distribution in it The multiple softwares or software module of formula service), single software or software module also may be implemented into.It is not specifically limited herein.

Server 105 can be to provide the server of various services, such as to 101,102,103 pairs of target persons of terminal device The image processing server that face carries out the Zuoren face image of shooting acquisition and right facial image is handled.Image processing server The data such as the Zuoren face image and right facial image received can be carried out the processing such as analyzing, and obtain processing result (such as Result three-dimensional grid corresponding to target face).

It should be noted that can be by server 105 for generating the method for information provided by embodiment of the disclosure It executes, can also be executed by terminal device 101,102,103；Correspondingly, it can be set for generating the device of information in service In device 105, also it can be set in terminal device 101,102,103.

It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented At the distributed server cluster that multiple servers form, individual server also may be implemented into.It, can when server is software It, can also be with to be implemented as multiple softwares or software module (such as providing multiple softwares of Distributed Services or software module) It is implemented as single software or software module.It is not specifically limited herein.

It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.Generating result three-dimensional grid corresponding to target face During used data do not need in the case where long-range obtain, above system framework can not include network, and only Including terminal device or server.

With continued reference to Fig. 2, the process of one embodiment of the method for generating information according to the disclosure is shown 200.The method for being used to generate information, comprising the following steps:

Step 201, it obtains and target face is carried out to shoot Zuoren face image obtained and right facial image.

In the present embodiment, can lead to for generating the executing subject (such as server shown in FIG. 1) of the method for information It crosses wired connection mode or radio connection acquisition carries out target face to shoot Zuoren face image obtained and right people Face image.Wherein, target face is the face of the three-dimensional grid to be generated corresponding to it.In practice, the three dimensional network of face is generated After lattice, three-dimensional grid can be carried out the operation such as rendering, and then realize three-dimensional facial reconstruction.

In the present embodiment, Zuoren face image and right facial image are binocular vision image.Specifically, above-mentioned executing subject It is available to be pre-stored within local Zuoren face image and right facial image, the also electronic equipment of available communication connection The Zuoren face image and right facial image that (such as terminal device shown in FIG. 1) is sent.It should be noted that Zuoren face image and Right facial image is two-dimensional facial image.

In practice, it can use the various equipment (such as binocular camera) including binocular camera and target face clapped It takes the photograph, obtains Zuoren face image and right facial image corresponding to target face.It should be noted that binocular camera is usually edge Two cameras of horizontal direction arrangement.When being shot using binocular camera, the camera in left side can will be arranged in It is determined as left camera, the image of shooting is left image (corresponding Zuoren face image)；It is corresponding, taking the photograph for right side will be arranged in As head is determined as right camera, the image of shooting is right image (corresponding right facial image).

Step 202, for the facial image in Zuoren face image and right facial image, following steps are executed: by the face Image input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image；For in mapping graph Point, the pixel value of coordinate and the point based on this in mapping graph, determines the three-dimensional coordinate of the corresponding face key point of the point； Based on the three-dimensional coordinate of face key point identified, in the facial image, three dimensional network corresponding to the facial image is generated Lattice.

In the present embodiment, for each of Zuoren face image and right facial image face figure obtained in step 201 Picture, above-mentioned executing subject can execute following steps:

Step 2021, facial image input mapping graph trained in advance is generated into model, it is right obtains facial image institute The mapping graph answered.

Wherein, mapping graph is the image for determining the three-dimensional coordinate of the face key point in facial image.Face is crucial The three-dimensional coordinate of point is made of the depth value of position coordinates of the face key point in facial image and face key point.Face The depth value of key point can be face key point when acquiring facial image to the distance of imaging plane.Point in mapping graph with Face key point in the facial image is corresponding.In practice, face key point can be point crucial in face, specifically, It can be the point of influence face mask or face shape.

In the present embodiment, mapping graph, which generates model, can be used for characterizing mapping corresponding to facial image and facial image The corresponding relationship of figure.Specifically, as an example, mapping graph generation model can be technical staff and be in advance based on to a large amount of face The statistics of mapping graph corresponding to image and facial image and pre-establish, be stored with multiple facial images and corresponding mapping The mapping table of figure；Or be based on preset training sample, using machine learning method to initial model (such as nerve Network) be trained after obtained model.

It should be noted that mapping graph generates the corresponding predetermined mapping relations of model or mapping principle, this is reflected Relationship or mapping principle are penetrated for determining that it is raw in mapping graph that input mapping graph generates the face key point in the facial image of model At the mapping position in the mapping graph of model output.

In some optional implementations of the present embodiment, mapping graph generate model can by above-mentioned executing subject or Other electronic equipments are obtained by following steps training:

Step 20211, training sample set is obtained.

Wherein, training sample includes the coordinate and depth of sample facial image, face key point in sample facial image Value.Here, sample facial image is two-dimensional facial image.In practice, training sample set can be obtained using various methods.

In some optional implementations of the present embodiment, the training sample that training sample is concentrated can pass through following step It is rapid to generate: it is possible, firstly, to using the face depth map of depth map acquisition device collecting sample face, and obtain face depth map Corresponding facial image.Then, face critical point detection is carried out to facial image corresponding to face depth map, to determine people The coordinate of face key point in facial image corresponding to face depth map.Finally, by face figure corresponding to face depth map Coordinate as, identified face key point, the depth value based on the face key point that face depth map determines summarize for training Sample.

Herein, depth map acquisition device can be it is various can be with the image collecting device of sampling depth figure.Such as binocular Camera, depth camera head etc..Face depth map is to include depth information (i.e. the range information on the surface of viewpoint and scenario objects) Image.Facial image corresponding to face depth map is the corresponding RGB (Red without depth information of face depth map Green Blue) Three Channel Color image.In turn, by removing the depth information of face depth map, face depth can be obtained The corresponding facial image (i.e. sample facial image) of figure.

Herein, various face critical point detection modes be can use, people is carried out to facial image corresponding to face depth map Face critical point detection.For example, facial image can be input to face critical point detection model trained in advance, detection knot is obtained Fruit.Wherein, face critical point detection model can be used for detecting the position of the face key point in facial image.Here, face Critical point detection model can be using machine learning method, (comprising facial image and be used to indicate face pass based on sample set The mark of the position of key point), what Training obtained is carried out to existing convolutional neural networks.Wherein, convolutional neural networks Various existing structures, such as DenseBox, VGGNet, ResNet, SegNet etc. can be used.

In addition, it is necessary to which explanation, the method for the depth value of the above-mentioned face key point determined based on face depth map are The well-known technique studied and applied extensively at present, details are not described herein again.

Step 20212, the training sample concentrated for training sample, the seat based on the face key point in the training sample Mark determines mapping position of the face key point in mapping graph to be built in the training sample, and is based on the training sample The depth value of face key point in this, determines the pixel value of corresponding mapping position in mapping graph to be built, using it is true The pixel value of fixed mapping position and mapping position constructs mapping graph corresponding with the sample facial image in the training sample.

Herein, the training sample concentrated for training sample, above-mentioned executing subject or other electronic equipments can be first Based on the coordinate of the face key point in the training sample, determine the face key point in the training sample in mapping to be built Mapping position in figure.It herein, can be based on the mapping relations pre-established or based on existing for some face key point Mapping principle determines the coordinate of the mapping position of the face key point in mapping graph to be built.As an example, can benefit The coordinate of the mapping position of the face key point in mapping graph to be built is determined with the principle that UV (U-VEEZ) maps.It is real In trampling, UV is 2 d texture coordinate.UV is for defining 2 d texture coordinate system, referred to as " UV texture space ".UV texture space makes The axis in two-dimensional space is indicated with letter U and V.In three-dimensional modeling, texture information can be converted to plane letter by UV mapping Breath.At this point, the UV coordinate mapped out can serve to indicate that the mapping position in mapping graph to be built.The UV mapped out is sat Mark can be used as the coordinate of the mapping position in mapping graph to be built.

In this implementation, determined some mapping position of face key point in mapping graph to be built it Afterwards, the pixel value of corresponding mapping position in mapping graph to be built can be determined based on the depth value of the face key point.Tool Body, it may be predetermined that the corresponding relationship of pixel value and depth value, in turn, based on above-mentioned corresponding relationship and the face key point Depth value, determine the pixel value of corresponding mapping position in mapping graph to be built.As an example, predetermined pixel value It is " pixel value is equal with depth value " with the corresponding relationship of depth value.And then for some face key point, the face key point Coordinate in sample facial image is (100,50), which is coordinate (50,25), the people The depth value of face key point is 30.Then the pixel value in mapping graph at coordinate (50,25) is 30.

It should be noted that the corresponding relationship of the size of mapping graph and the size of sample facial image can be pre-established. As an example, the size that can preset mapping graph is identical as the size of sample facial image；It is reflected alternatively, can preset The size of figure is penetrated as the half of the size of sample facial image.

Step 20213, using machine learning method, the sample facial image of the training sample that training sample is concentrated as Input, using mapping graph corresponding to the sample facial image inputted as desired output, training obtains mapping graph and generates model.

Herein, above-mentioned executing subject or other electronic equipments can use machine learning method, by above-mentioned training sample Input of the sample facial image that the training sample of concentration includes as initial model, the sample facial image institute inputted is right Desired output of the mapping graph answered as initial model, is trained initial model, and final training obtains mapping graph and generates mould Type.

Herein, can be used various existing convolutional neural networks structures (such as DenseBox, VGGNet, ResNet, SegNet etc.) it is trained as initial model.In practice, convolutional neural networks (Convolutional Neural Network, CNN) it is a kind of feedforward neural network, its artificial neuron can respond single around in a part of coverage area Member has outstanding performance for image procossing, therefore, it is possible to using convolutional neural networks to the sample facial image in training sample It is handled.

It should be noted that other also can be used with image processing function in above-mentioned executing subject or other electronic equipments Model as initial model, however it is not limited to CNN, specific model structure can set according to actual needs, not limit herein It is fixed.It should be pointed out that machine learning method is the well-known technique studied and applied extensively at present, details are not described herein.

Step 2022, for the point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph is determined The three-dimensional coordinate of the corresponding face key point of the point.

Herein, for the point in mapping graph, above-mentioned executing subject can be based on the point in mapping graph using various methods In coordinate and the point pixel value, determine the three-dimensional coordinate of the corresponding face key point of the point.

In some optional implementations of the present embodiment, for the point in mapping graph, above-mentioned executing subject can lead to Cross the three-dimensional coordinate that following steps determine the corresponding face key point of the point: firstly, above-mentioned executing subject can be based on the point Pixel value determines the depth value of the corresponding face key point of the point.Then, above-mentioned executing subject can be based on the point in mapping graph In coordinate determine coordinate of the corresponding face key point of the point in the facial image.Finally, above-mentioned executing subject can be with base In coordinate of the depth value of the corresponding face key point of the point face key point corresponding with the point in the facial image, determine The three-dimensional coordinate of the corresponding face key point of the point.

Specifically, above-mentioned executing subject can generate the corresponding mapping relations of model or mapping principle based on mapping graph, Coordinate of the corresponding face key point of the point in the facial image is determined based on coordinate of this in mapping graph.It is understood that , when training mapping graph generates model, due to utilizing predetermined mapping relations or mapping principle, face can be based on The coordinate of key point determines mapping position of the face key point in mapping graph to be built (with reference to step 20212).Thus, Herein, for some point in mapping graph, reverse process can be used, determines the corresponding face key point of the point in facial image In coordinate.

In addition, above-mentioned executing subject can be determined based on the pixel value of the point using various methods in this implementation The depth value of the corresponding face key point of the point.For example, the pixel value of the point directly can be determined as the corresponding face of point The depth value of key point.

In some optional implementations of the present embodiment, above-mentioned executing subject can be in response to the pixel of the determining point Value is more than or equal to preset threshold, and the pixel value of the point is determined as to the depth value of the corresponding face key point of the point.Preset threshold It can be predetermined value, such as " 1 ".In practice, the very low point of pixel value is usually the point for predicting fault, therefore in this reality In existing mode, the point of prediction fault can be removed by the way that preset threshold is arranged, aids in determining whether out that more accurate face is crucial The three-dimensional coordinate of point.

In some optional implementations of the present embodiment, above-mentioned executing subject may also respond to determine the picture of the point Element value is less than preset threshold, and preset threshold is determined as to the depth value of the corresponding face key point of the point.

It should be noted that the depth value of coordinate and face key point of the face key point in facial image has been determined Afterwards, above-mentioned executing subject can directly utilize the depth of coordinate (can be expressed as (x, y)) and face key point of face key point The three-dimensional coordinate ((x, y, z) can be expressed as) of angle value (z can be expressed as) composition face key point.

Step 2023, based on the three-dimensional coordinate of face key point identified, in the facial image, the face figure is generated As corresponding three-dimensional grid.

In practice, three-dimensional grid corresponding to facial image be using face key point as the three-dimensional grid on vertex, therefore, Here, the three-dimensional coordinate based on each face key point determine, in the facial image, above-mentioned executing subject can be generated Three-dimensional grid corresponding to the facial image.

It should be noted that the three-dimensional coordinate on the vertex based on three-dimensional grid, the method for generating three-dimensional grid is wide at present The well-known technique of general research and application, details are not described herein again.

Step 203, raw based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image At result three-dimensional grid corresponding to target face.

In the present embodiment, above-mentioned executing subject can be generated three corresponding to Zuoren face image by executing step 202 Tie up three-dimensional grid corresponding to grid and right facial image.In turn, based on three-dimensional grid corresponding to Zuoren face image and right people Result three-dimensional grid corresponding to target face can be generated in three-dimensional grid corresponding to face image, above-mentioned executing subject.Wherein, As a result three-dimensional grid is to carry out the operation such as rendering to it, to realize the three dimensional network of the three-dimensional facial reconstruction for target face Lattice.

Specifically, above-mentioned executing subject can be right based on three-dimensional grid corresponding to Zuoren face image and right facial image institute The three-dimensional grid answered generates result three-dimensional grid corresponding to target face using various methods.For example, can detecte Zuoren face The head pose of three-dimensional grid corresponding to the head pose of three-dimensional grid corresponding to image and right facial image, in turn, by institute Three-dimensional grid corresponding to the lesser facial image of end rotation angle indicated by corresponding head pose is determined as target person Result three-dimensional grid corresponding to face.It is appreciated that end rotation angle is smaller, and the face characteristic blocked is fewer in practice, it is raw At three-dimensional grid it is then more accurate.To three corresponding to three-dimensional grid and right facial image corresponding to the Zuoren face image The corresponding lesser three-dimensional grid of end rotation angle is chosen in dimension grid as result three dimensional network corresponding to target face Lattice help to improve the accuracy of result three-dimensional grid.

It should be noted that the method for detection head pose is the well-known technique studied and applied extensively at present, herein not It repeats again.

With continued reference to the signal that Fig. 3, Fig. 3 are according to the application scenarios of the method for generating information of the present embodiment Figure.It is that the available terminal device 302 first of server 301 is sent, target face is clapped in the application scenarios of Fig. 3 Take the photograph Zuoren face image 303 obtained and right facial image 304, wherein Zuoren face image 303 and right facial image 304 are double Mesh visual pattern.

Then, for Zuoren face image 303, Zuoren face image 303 can be inputted mapping trained in advance by server 301 Figure generates model 305, obtains mapping graph 306 corresponding to Zuoren face image 303, wherein point and Zuoren face in mapping graph 306 Face key point in image 303 is corresponding；For the point in mapping graph 306, coordinate based on this in mapping graph 306 and The pixel value of the point determines the three-dimensional coordinate of the corresponding face key point of the point；Based in identified, Zuoren face image 303 Face key point three-dimensional coordinate, generate Zuoren face image 303 corresponding to three-dimensional grid 307；For right facial image 304, right facial image 304 can be inputted mapping graph and generate model 305 by server 301, be obtained corresponding to right facial image 304 Mapping graph 308, wherein the point in mapping graph 308 is corresponding with the face key point in right facial image 304；For mapping Point in Figure 30 8, the pixel value of coordinate and the point based on this in mapping graph 308 determine that the corresponding face of the point is crucial The three-dimensional coordinate of point；Based on the three-dimensional coordinate of the face key point in identified, right facial image 304, right face figure is generated The three-dimensional grid 309 as corresponding to 304.

Finally, server 301 can be based on three-dimensional grid 307 corresponding to Zuoren face image 303 and right facial image 304 Corresponding three-dimensional grid 309 generates result three-dimensional grid 310 corresponding to target face.

The method provided by the above embodiment of the disclosure carries out target face to shoot Zuoren face obtained by obtaining Image and right facial image execute following steps: by this then for the facial image in Zuoren face image and right facial image Facial image input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image；For mapping graph In point, the pixel value of coordinate based on this in mapping graph and the point determines the three-dimensional of the corresponding face key point of the point Coordinate；Based on the three-dimensional coordinate of face key point identified, in the facial image, generate three corresponding to the facial image Grid is tieed up, finally based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, generates mesh Mark result three-dimensional grid corresponding to face.It is appreciated that due to blocking with shooting angle etc., Zuoren face image and right people Face image can recorde the face characteristic of different angle, so here, utilizing three-dimensional grid corresponding to Zuoren face image and the right side Three-dimensional grid corresponding to facial image can be generated result three-dimensional grid corresponding to more accurate, target face, help In the accuracy for improving three-dimensional facial reconstruction.

With further reference to Fig. 4, it illustrates the processes 400 of another embodiment of the method for generating information.The use In the process 400 for the method for generating information, comprising the following steps:

Step 401, it obtains and target face is carried out to shoot Zuoren face image obtained and right facial image.

In the present embodiment, can lead to for generating the executing subject (such as server shown in FIG. 1) of the method for information It crosses wired connection mode or radio connection acquisition carries out target face to shoot Zuoren face image obtained and right people Face image.Wherein, target face is the face of the three-dimensional grid to be generated corresponding to it.In practice, the three dimensional network of face is generated After lattice, three-dimensional grid can be carried out the operation such as rendering, and then realize three-dimensional facial reconstruction.Zuoren face image and right facial image For binocular vision image.

Step 402, for the facial image in Zuoren face image and right facial image, following steps are executed: by the face Image input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image；For in mapping graph Point, the pixel value of coordinate and the point based on this in mapping graph, determines the three-dimensional coordinate of the corresponding face key point of the point； Based on the three-dimensional coordinate of face key point identified, in the facial image, three dimensional network corresponding to the facial image is generated Lattice.

In the present embodiment, for each of Zuoren face image and right facial image face figure obtained in step 401 Picture, above-mentioned executing subject can execute following steps:

Step 4021, facial image input mapping graph trained in advance is generated into model, it is right obtains facial image institute The mapping graph answered.

Wherein, mapping graph is the image for determining the three-dimensional coordinate of the face key point in facial image.Face is crucial The three-dimensional coordinate of point is made of the depth value of position coordinates of the face key point in facial image and face key point.Face The depth value of key point can be face key point when acquiring facial image to the distance of imaging plane.Point in mapping graph with Face key point in the facial image is corresponding.In practice, face key point can be point crucial in face, specifically, It can be the point of influence face mask or face shape.Mapping graph, which generates model, can be used for characterizing facial image and face figure As the corresponding relationship of corresponding mapping graph.

Step 4022, for the point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph is determined The three-dimensional coordinate of the corresponding face key point of the point.

Step 4023, based on the three-dimensional coordinate of face key point identified, in the facial image, the face figure is generated As corresponding three-dimensional grid.

Above-mentioned steps 401, step 402 are consistent with step 201, the step 202 in previous embodiment respectively, above with respect to step Rapid 201, the description of step 202 is also applied for step 401, step 402, and details are not described herein again.

Step 403, corresponding to the center line and right facial image for crossing three-dimensional grid corresponding to Zuoren face image respectively The center line of three-dimensional grid, establishes datum level.

In the present embodiment, above-mentioned executing subject can be generated three corresponding to Zuoren face image by executing step 402 Tie up three-dimensional grid corresponding to grid and right facial image.In turn, it is right can to cross Zuoren face image institute respectively for above-mentioned executing subject The center line of three-dimensional grid corresponding to the center line for the three-dimensional grid answered and right facial image, establishes datum level.Wherein, benchmark Three-dimensional grid is run through along center line in face, and three-dimensional grid is divided into two parts.Specifically, two parts for being divided of datum level can be with For symmetrical two parts (axis of symmetry that datum level crosses face at this time), or asymmetric two parts.But it needs bright True, datum level can be by a point on face indicated by three-dimensional grid.In turn, datum level can be by three-dimensional grid Indicated face is divided into the face on the left of datum level and the face on the right side of datum level.In addition, being directed to Zuoren face The point that three-dimensional grid corresponding to image is established on the facial image that is passed through of datum level with for corresponding to right facial image The facial image that is passed through of datum level established of three-dimensional grid on the point assignor same point on the face (such as indicate nose Point corresponding to point).

Step 404, it extracts in three-dimensional grid corresponding to Zuoren face image, the three-dimensional grid conduct on the left of datum level Left three-dimensional grid, and extract in three-dimensional grid corresponding to right facial image, the three-dimensional grid conduct on the right side of datum level Right three-dimensional grid.

In the present embodiment, based on two datum levels established in step 403, above-mentioned executing subject can extract Zuoren face In three-dimensional grid corresponding to image, the three-dimensional grid on the left of datum level is as left three-dimensional grid, and extracts right face In three-dimensional grid corresponding to image, the three-dimensional grid on the right side of datum level is as right three-dimensional grid.

It should be noted that the three-dimensional grid being located on the left of datum level is towards in the three-dimensional grid for establishing datum level When facial contour, the three-dimensional grid in the left side of datum level；Three-dimensional grid on the right side of datum level is towards establishing datum level Three-dimensional grid in facial contour when, the three-dimensional grid on the right side of datum level.

Step 405, extracted left three-dimensional grid and right three-dimensional grid are spliced, is generated corresponding to target face As a result three-dimensional grid.

In the present embodiment, based on left three-dimensional grid and right three-dimensional grid obtained in step 404, above-mentioned executing subject can To splice to left three-dimensional grid and right three-dimensional grid, result three-dimensional grid corresponding to target face is generated.

It should be noted that since datum level crosses the center line of three-dimensional grid, and for three corresponding to Zuoren face image The point on facial image that the datum level that dimension grid is established is passed through is established with for three-dimensional grid corresponding to right facial image The facial image that is passed through of datum level on the point assignor same point on the face, so three-dimensional corresponding to Zuoren face image In grid, the three-dimensional grid on the left of datum level can be located at datum level with three-dimensional grid corresponding to right facial image The three-dimensional grid on right side is spliced into three-dimensional grid corresponding to a complete face；Three-dimensional grid corresponding to Zuoren face image In, the three-dimensional grid on the right side of datum level can be located on the left of datum level with three-dimensional grid corresponding to right facial image Three-dimensional grid be spliced into three-dimensional grid corresponding to a complete face.

And in practicing, it is special that Zuoren face image can recorde face corresponding to more three-dimensional grids on the left of datum level Sign, right facial image can recorde face characteristic corresponding to more three-dimensional grids on the right side of datum level, so, in this reality It applies in example, extracts three dimensional network corresponding to the left three-dimensional grid and right facial image in three-dimensional grid corresponding to Zuoren face image Right three-dimensional grid in lattice, and extracted left three-dimensional grid and right three-dimensional grid are spliced, it is right to generate target face institute The result three-dimensional grid answered, can make result three-dimensional grid generated more precisely characterize the face characteristic of target face, Improve the accuracy of result three-dimensional grid generated.

Figure 4, it is seen that the method for generating information compared with the corresponding embodiment of Fig. 2, in the present embodiment Process 400 highlight and extract left three-dimensional grid from three-dimensional grid corresponding to Zuoren face image, it is right from right facial image institute Right three-dimensional grid is extracted in the three-dimensional grid answered, and then left three-dimensional grid and right three-dimensional grid are spliced, and generates target person Corresponding to face the step of result three-dimensional grid.It is appreciated that Zuoren face image can recorde more multidigit due to blocking etc. Face characteristic corresponding to three-dimensional grid on the left of the datum level, and right facial image can recorde and more be located on the right side of datum level Three-dimensional grid corresponding to face characteristic, so the present embodiment description scheme, utilize a left side three corresponding to Zuoren face image Right three-dimensional grid corresponding to grid and right facial image is tieed up, result corresponding to more accurate, target face can be generated Three-dimensional grid facilitates the accuracy for further increasing three-dimensional facial reconstruction.

With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, present disclose provides one kind for generating letter One embodiment of the device of breath, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, which can specifically answer For in various electronic equipments.

As shown in figure 5, the device 500 for generating information of the present embodiment includes: that image acquisition unit 501, first is raw At unit 502 and the second generation unit 503.Wherein, image acquisition unit 501 is configured to obtain and shoot to target face Zuoren face image obtained and right facial image, wherein Zuoren face image and right facial image are binocular vision image；First Generation unit 502 is configured to execute following steps: by the people for the facial image in Zuoren face image and right facial image Face image input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image, wherein in mapping graph Point it is corresponding with the face key point in the facial image；Seat for the point in mapping graph, based on this in mapping graph The pixel value of mark and the point, determines the three-dimensional coordinate of the corresponding face key point of the point；Based in the identified, facial image Face key point three-dimensional coordinate, generate three-dimensional grid corresponding to the facial image；Second generation unit 503 is configured to Based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, generate corresponding to target face Result three-dimensional grid.

It in the present embodiment, can be by wired connection side for generating the image acquisition unit 501 of the device 500 of information Formula or radio connection acquisition carry out target face to shoot Zuoren face image obtained and right facial image.Wherein, Target face is the face of the three-dimensional grid to be generated corresponding to it.Zuoren face image and right facial image are binocular vision figure Picture.

In the present embodiment, each of the Zuoren face image obtained for image acquisition unit 501 and right facial image Facial image, the first generation unit 502 can execute following steps: facial image input mapping graph trained in advance is generated Model obtains mapping graph corresponding to the facial image；For the point in mapping graph, coordinate based on this in mapping graph and The pixel value of the point determines the three-dimensional coordinate of the corresponding face key point of the point；Based on people identified, in the facial image The three-dimensional coordinate of face key point generates three-dimensional grid corresponding to the facial image.

In the present embodiment, mapping graph, which generates model, can be used for characterizing mapping corresponding to facial image and facial image The corresponding relationship of figure.It should be noted that mapping graph generates the corresponding predetermined mapping relations of model or mapping principle, The mapping relations or mapping principle are being mapped for determining that input mapping graph generates the face key point in the facial image of model Figure generates the mapping position in the mapping graph of model output.

In the present embodiment, three-dimensional grid and the right side corresponding to the Zuoren face image obtained based on the first generation unit 502 Result three-dimensional grid corresponding to target face can be generated in three-dimensional grid corresponding to facial image, the second generation unit 503. Wherein, as a result three-dimensional grid is to carry out the operation such as rendering to it, to realize three of the three-dimensional facial reconstruction for target face Tie up grid.

In some optional implementations of the present embodiment, the first generation unit 502 can be further configured to: base In the pixel value of the point, the depth value of the corresponding face key point of the point is determined；It is determined based on coordinate of this in mapping graph Coordinate of the corresponding face key point of the point in the facial image；Depth value based on the corresponding face key point of the point and should Coordinate of the corresponding face key point of point in the facial image, determines the three-dimensional coordinate of the corresponding face key point of the point.

In some optional implementations of the present embodiment, the first generation unit 502 can be further configured to: be rung Preset threshold should be more than or equal in determining the pixel value of the point, the pixel value of the point is determined as the corresponding face key point of the point Depth value.

In some optional implementations of the present embodiment, the first generation unit 502 can be further configured to: be rung Preset threshold should be less than in determining the pixel value of the point, preset threshold is determined as to the depth of the corresponding face key point of the point Value.

In some optional implementations of the present embodiment, the second generation unit 503 may include: to establish module (figure In be not shown), be configured to respectively cross Zuoren face image corresponding to three-dimensional grid center line and right facial image corresponding to Three-dimensional grid center line, establish datum level, wherein datum level along center line run through three-dimensional grid, three-dimensional grid is divided At two parts；Extraction module (not shown) is configured to extract in three-dimensional grid corresponding to Zuoren face image, is located at base The three-dimensional grid of quasi- left side of face is as left three-dimensional grid, and extracts in three-dimensional grid corresponding to right facial image, is located at base The three-dimensional grid of quasi- right side of face is as right three-dimensional grid；Splicing module (not shown) is configured to extracted left three Dimension grid and right three-dimensional grid are spliced, and result three-dimensional grid corresponding to target face is generated.

In some optional implementations of the present embodiment, mapping graph generates model and can be obtained by following steps training : obtain training sample set, wherein training sample includes sample facial image, the face key point in sample facial image Coordinate and depth value；For the training sample that training sample is concentrated, based on the coordinate of the face key point in the training sample, really Mapping position of the face key point in mapping graph to be built in the fixed training sample, and based in the training sample The depth value of face key point is determined the pixel value of corresponding mapping position in mapping graph to be built, is reflected using identified The pixel value of position and mapping position is penetrated, mapping graph corresponding with the sample facial image in the training sample is constructed；Utilize machine Device learning method, the sample facial image for the training sample that training sample is concentrated is as input, the sample face that will be inputted Mapping graph corresponding to image obtains mapping graph and generates model as desired output, training.

In some optional implementations of the present embodiment, the training sample that training sample is concentrated can pass through following step It is rapid to generate: using the face depth map of depth map acquisition device collecting sample face, and to obtain corresponding to face depth map Facial image；Face critical point detection is carried out to facial image corresponding to face depth map, to determine that face depth map institute is right The coordinate of face key point in the facial image answered；Facial image corresponding to face depth map, identified face are closed The coordinate of key point, the depth value of face key point determined based on face depth map are summarized for training sample.

It is understood that all units recorded in the device 500 and each step phase in the method with reference to Fig. 2 description It is corresponding.As a result, above with respect to the operation of method description, the beneficial effect of feature and generation be equally applicable to device 500 and its In include unit, details are not described herein.

The device provided by the above embodiment 500 of the disclosure carries out target face to shoot Zuoren obtained by obtaining Face image and right facial image execute following steps then for the facial image in Zuoren face image and right facial image: will Facial image input mapping graph trained in advance generates model, obtains mapping graph corresponding to the facial image；For mapping Point in figure, the pixel value of coordinate and the point based on this in mapping graph, determines the three of the corresponding face key point of the point Tie up coordinate；Based on the three-dimensional coordinate of face key point identified, in the facial image, generate corresponding to the facial image Three-dimensional grid is generated finally based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image Result three-dimensional grid corresponding to target face.It is appreciated that due to blocking with shooting angle etc., Zuoren face image and the right side Facial image can recorde the face characteristic of different angle, so here, using three-dimensional grid corresponding to Zuoren face image and Three-dimensional grid corresponding to right facial image can be generated result three-dimensional grid corresponding to more accurate, target face, have Help improve the accuracy of three-dimensional facial reconstruction.

Below with reference to Fig. 6, it illustrates the electronic equipment that is suitable for being used to realize embodiment of the disclosure, (example is as shown in figure 1 Server or terminal device) 600 structural schematic diagram.Terminal device in embodiment of the disclosure can include but is not limited to all As mobile phone, laptop, digit broadcasting receiver, PDA (personal digital assistant), PAD (tablet computer), PMP are (portable Formula multimedia player), the mobile terminal and such as number TV, desk-top meter of car-mounted terminal (such as vehicle mounted guidance terminal) etc. The fixed terminal of calculation machine etc..Terminal device or server shown in Fig. 6 are only an example, should not be to the implementation of the disclosure The function and use scope of example bring any restrictions.

As shown in fig. 6, electronic equipment 600 may include processing unit (such as central processing unit, graphics processor etc.) 601, random access can be loaded into according to the program being stored in read-only memory (ROM) 602 or from storage device 608 Program in memory (RAM) 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with electronic equipment Various programs and data needed for 600 operations.Processing unit 601, ROM 602 and RAM603 are connected with each other by bus 604. Input/output (I/O) interface 605 is also connected to bus 604.

In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, taking the photograph As the input unit 606 of head, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaker, vibration The output device 607 of dynamic device etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.Communication device 609, which can permit electronic equipment 600, is wirelessly or non-wirelessly communicated with other equipment to exchange data.Although Fig. 6 shows tool There is the electronic equipment 600 of various devices, it should be understood that being not required for implementing or having all devices shown.It can be with Alternatively implement or have more or fewer devices.Each box shown in Fig. 6 can represent a device, can also root According to needing to represent multiple devices.

Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium On computer program, which includes the program code for method shown in execution flow chart.In such reality It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608 It is mounted, or is mounted from ROM 602.When the computer program is executed by processing unit 601, the implementation of the disclosure is executed The above-mentioned function of being limited in the method for example.It should be noted that computer-readable medium described in embodiment of the disclosure can be with It is computer-readable signal media or computer readable storage medium either the two any combination.It is computer-readable Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or Device, or any above combination.The more specific example of computer readable storage medium can include but is not limited to: have The electrical connection of one or more conducting wires, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD- ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In embodiment of the disclosure, computer Readable storage medium storing program for executing can be any tangible medium for including or store program, which can be commanded execution system, device Either device use or in connection.And in embodiment of the disclosure, computer-readable signal media may include In a base band or as the data-signal that carrier wave a part is propagated, wherein carrying computer-readable program code.It is this The data-signal of propagation can take various forms, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate Combination.Computer-readable signal media can also be any computer-readable medium other than computer readable storage medium, should Computer-readable signal media can send, propagate or transmit for by instruction execution system, device or device use or Person's program in connection.The program code for including on computer-readable medium can transmit with any suitable medium, Including but not limited to: electric wire, optical cable, RF (radio frequency) etc. or above-mentioned any appropriate combination.

Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not It is fitted into the electronic equipment.Above-mentioned computer-readable medium carries one or more program, when said one or more When a program is executed by the electronic equipment, so that the electronic equipment: acquisition carries out target face to shoot Zuoren face obtained Image and right facial image, wherein Zuoren face image and right facial image are binocular vision image；For Zuoren face image and the right side Facial image in facial image executes following steps: facial image input mapping graph trained in advance being generated model, is obtained Obtain mapping graph corresponding to the facial image, wherein the point in mapping graph is corresponding with the face key point in the facial image； For the point in mapping graph, the pixel value of coordinate and the point based on this in mapping graph determines that the corresponding face of the point closes The three-dimensional coordinate of key point；Based on the three-dimensional coordinate of face key point identified, in the facial image, the facial image is generated Corresponding three-dimensional grid；Based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, Generate result three-dimensional grid corresponding to target face.

The behaviour for executing embodiment of the disclosure can be write with one or more programming languages or combinations thereof The computer program code of work, described program design language include object oriented program language-such as Java, Smalltalk, C++ further include conventional procedural programming language-such as " C " language or similar program design language Speech.Program code can be executed fully on the user computer, partly be executed on the user computer, as an independence Software package execute, part on the user computer part execute on the remote computer or completely in remote computer or It is executed on server.In situations involving remote computers, remote computer can pass through the network of any kind --- packet It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit It is connected with ISP by internet).

Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction Combination realize.

Being described in unit involved in embodiment of the disclosure can be realized by way of software, can also be passed through The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor Including image acquisition unit, the first generation unit and the second generation unit.Wherein, the title of these units is under certain conditions simultaneously The restriction to the unit itself is not constituted, for example, image acquisition unit is also described as " obtaining the unit of facial image ".

Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art Member it should be appreciated that embodiment of the disclosure involved in invention scope, however it is not limited to the specific combination of above-mentioned technical characteristic and At technical solution, while should also cover do not depart from foregoing invention design in the case where, by above-mentioned technical characteristic or its be equal Feature carries out any combination and other technical solutions for being formed.Such as disclosed in features described above and embodiment of the disclosure (but It is not limited to) technical characteristic with similar functions is replaced mutually and the technical solution that is formed.

Claims

1. a kind of method for generating information, comprising:

Acquisition carries out target face to shoot Zuoren face image obtained and right facial image, wherein Zuoren face image and the right side Facial image is binocular vision image；

For the facial image in Zuoren face image and right facial image, following steps are executed: the facial image being inputted preparatory Trained mapping graph generates model, obtains mapping graph corresponding to the facial image, wherein point and the face figure in mapping graph Face key point as in is corresponding；For the point in mapping graph, the pixel of coordinate and the point based on this in mapping graph Value, determines the three-dimensional coordinate of the corresponding face key point of the point；Based on face key point identified, in the facial image Three-dimensional coordinate generates three-dimensional grid corresponding to the facial image；

Based on three-dimensional grid corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image, the target person is generated Result three-dimensional grid corresponding to face.

2. according to the method described in claim 1, wherein, the pixel of the coordinate based on this in mapping graph and the point Value, determines the three-dimensional coordinate of the corresponding face key point of the point, comprising:

Based on the pixel value of the point, the depth value of the corresponding face key point of the point is determined；

Coordinate of the corresponding face key point of the point in the facial image is determined based on coordinate of this in mapping graph；

Seat of the depth value face key point corresponding with the point in the facial image based on the corresponding face key point of the point Mark, determines the three-dimensional coordinate of the corresponding face key point of the point.

3. according to the method described in claim 2, wherein, the pixel value based on the point determines that the corresponding face of the point closes The depth value of key point, comprising:

In response to determining that the pixel value of the point is more than or equal to preset threshold, the pixel value of the point is determined as the corresponding face of point The depth value of key point.

4. according to the method described in claim 3, wherein, the pixel value based on the point determines that the corresponding face of the point closes The depth value of key point, further includes:

In response to determining that the pixel value of the point is less than the preset threshold, the preset threshold is determined as the corresponding face of point The depth value of key point.

5. described based on three-dimensional grid corresponding to Zuoren face image and right face according to the method described in claim 1, wherein Three-dimensional grid corresponding to image generates result three-dimensional grid corresponding to the target face, comprising:

It is crossed in three-dimensional grid corresponding to the center line and right facial image of three-dimensional grid corresponding to Zuoren face image respectively Heart line, establishes datum level, wherein datum level runs through three-dimensional grid along center line, and three-dimensional grid is divided into two parts；

Three-dimensional grid corresponding to Zuoren face image in three-dimensional grid, on the left of datum level is extracted as left three-dimensional grid, And extract in three-dimensional grid corresponding to right facial image, the three-dimensional grid on the right side of datum level is as right three-dimensional grid；

Extracted left three-dimensional grid and right three-dimensional grid are spliced, the three-dimensional of result corresponding to the target face is generated Grid.

6. method described in one of -5 according to claim 1, wherein the mapping graph generates model and obtained by following steps training :

Obtain training sample set, wherein training sample includes sample facial image, the face key point in sample facial image Coordinate and depth value；

The training sample concentrated for training sample determines the training based on the coordinate of the face key point in the training sample Mapping position of the face key point in mapping graph to be built in sample, and it is crucial based on the face in the training sample Point depth value, determine the pixel value of corresponding mapping position in mapping graph to be built, using identified mapping position with The pixel value of mapping position constructs mapping graph corresponding with the sample facial image in the training sample；

Using machine learning method, the sample facial image for the training sample that training sample is concentrated will be inputted as input Sample facial image corresponding to mapping graph as desired output, training obtains mapping graph and generates model.

7. according to the method described in claim 6, wherein, the training sample that the training sample is concentrated is raw by following steps At:

Using the face depth map of depth map acquisition device collecting sample face, and obtain corresponding to the face depth map Facial image；

Face critical point detection is carried out to facial image corresponding to the face depth map, with the determination face depth map institute The coordinate of face key point in corresponding facial image；

By facial image corresponding to the face depth map, the coordinate of identified face key point, based on face depth map The depth value of determining face key point summarizes for training sample.

8. a kind of for generating the device of information, comprising:

Image acquisition unit is configured to acquisition and carries out shooting Zuoren face image obtained and right face figure to target face Picture, wherein Zuoren face image and right facial image are binocular vision image；

First generation unit is configured to execute following steps for the facial image in Zuoren face image and right facial image: Facial image input mapping graph trained in advance is generated into model, obtains mapping graph corresponding to the facial image, wherein reflect It is corresponding with the face key point in the facial image to penetrate the point in figure；For the point in mapping graph, based on the point in mapping graph In coordinate and the point pixel value, determine the three-dimensional coordinate of the corresponding face key point of the point；Based on the identified, face The three-dimensional coordinate of face key point in image, generates three-dimensional grid corresponding to the facial image；

Second generation unit is configured to based on three corresponding to three-dimensional grid corresponding to Zuoren face image and right facial image Grid is tieed up, result three-dimensional grid corresponding to the target face is generated.

9. device according to claim 8, wherein first generation unit is further configured to:

10. device according to claim 9, wherein first generation unit is further configured to:

11. device according to claim 10, wherein first generation unit is further configured to:

12. device according to claim 8, wherein second generation unit includes:

Module is established, center line and the right facial image institute for being configured to cross three-dimensional grid corresponding to Zuoren face image respectively are right The center line for the three-dimensional grid answered, establishes datum level, wherein datum level runs through three-dimensional grid along center line, and three-dimensional grid is drawn It is divided into two parts；

Extraction module is configured to extract the three dimensional network in three-dimensional grid corresponding to Zuoren face image, on the left of datum level Lattice are as left three-dimensional grid, and extract in three-dimensional grid corresponding to right facial image, the three dimensional network on the right side of datum level Lattice are as right three-dimensional grid；

Splicing module is configured to splice extracted left three-dimensional grid and right three-dimensional grid, generates the target person Result three-dimensional grid corresponding to face.

13. the device according to one of claim 8-12, wherein the mapping graph generates model and passes through following steps training It obtains:

14. device according to claim 13, wherein the training sample that the training sample is concentrated is raw by following steps At:

15. a kind of electronic equipment, comprising:

One or more processors；

Storage device is stored thereon with one or more programs,

When one or more of programs are executed by one or more of processors, so that one or more of processors are real The now method as described in any in claim 1-7.

16. a kind of computer-readable medium, is stored thereon with computer program, wherein the realization when program is executed by processor Method as described in any in claim 1-7.