CN108921795A

CN108921795A - A kind of image interfusion method, device and storage medium

Info

Publication number: CN108921795A
Application number: CN201810563880.1A
Authority: CN
Inventors: 周景锦; 程培; 钱梦仁; 傅斌
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-06-04
Filing date: 2018-06-04
Publication date: 2018-11-30
Also published as: WO2019233229A1

Abstract

The embodiment of the invention discloses a kind of image interfusion method, device and storage medium, the embodiment of the present invention obtains two dimensional image to be fused, and constructs three-dimensional face model according to the face in the two dimensional image to be fused；Template image is obtained, and obtains the rotation parameter of face in three dimensions in the template image；The three-dimensional face model is rotated based on the rotation parameter, three-dimensional face model after being rotated；Human face three-dimensional model after the rotation is converted into two-dimension human face image；The two-dimension human face image is merged with the template image.The program passes through building and rotated three dimensional faceform, so as to obtain two-dimension human face image based on three-dimensional face model after rotation, so that two-dimension human face image is consistent with the direction of face in template image, so that the face in two-dimension human face image to be precisely fused to the face region in template image, the effect merged to face in image is improved.

Description

A kind of image interfusion method, device and storage medium

Technical field

The present invention relates to technical field of image processing, and in particular to a kind of image interfusion method, device and storage medium.

Background technique

Image fusion technology is one of computer vision key technology, and have a wide range of applications scene, for example, changing Face, average face, mad Face Changing etc. are greatly favored by consumers, and key technology is exactly face fusion technology.

In the prior art, face fusion is fused to mainly by the face area in the facial image comprising user's face In Prototype drawing comprising face, to replace the face in Prototype drawing, for example, the pixel of the face area in facial image is moved Move on to corresponding face area in Prototype drawing.The program only Prototype drawing it is consistent with the face direction in facial image (such as All be positive face or be all side face) in the case where can just access preferable syncretizing effect, and people in the facial image that user takes pictures Face direction be extremely difficult to it is consistent in Prototype drawing, for example, if allow user clap one include side face facial image, it is intended to keep With side face posture consistent in Prototype drawing, then during taking pictures, the posture and angle of user's side face are extremely difficult to and template Side face in figure is consistent.And when the face direction in Prototype drawing and facial image is inconsistent, in Prototype drawing and facial image Human face characteristic point has relatively large deviation, can only be merged by force.For example, the face in the Prototype drawing is side face, and face figure It is sightless since the side face in Prototype drawing has partial region, for example be blocked side nose when face as in is positive face, And all positive face regions in facial image can be all fused in Prototype drawing, at this point, in side face and facial image in Prototype drawing Positive face between human face characteristic point have deviation, carry out by force fusion will cause distortion and offset etc..

In the research and practice process to the prior art, it was found by the inventors of the present invention that due to when need Prototype drawing and When face direction in facial image is consistent, face direction (such as posture and angle) is difficult to reach in the facial image that user provides The effect is relatively poor for the face fusion for being consistent, therefore obtaining to Prototype drawing.And when the face in Prototype drawing and facial image When direction inconsistent (such as the face in Prototype drawing is side face, and the face in facial image is positive face), merged by force It will cause distortion and offset equal error, cause face fusion effect excessively poor.

Summary of the invention

The embodiment of the present invention provides a kind of image interfusion method, device and storage medium, it is intended to improve to face in image The effect merged.

In order to solve the above technical problems, the embodiment of the present invention provides following technical scheme：

A kind of image interfusion method, including：

Two dimensional image to be fused is obtained, and three-dimensional face model is constructed according to the face in the two dimensional image to be fused；

Template image is obtained, and obtains the rotation parameter of face in three dimensions in the template image；

The three-dimensional face model is rotated based on the rotation parameter, three-dimensional face model after being rotated；

Human face three-dimensional model after the rotation is converted into two-dimension human face image；

The two-dimension human face image is merged with the template image.

A kind of image fusion device, including：

Construction unit is constructed for obtaining two dimensional image to be fused, and according to the face in the two dimensional image to be fused Three-dimensional face model；

Acquiring unit for obtaining template image, and obtains face in the template image in three dimensions Rotation parameter；

Rotary unit, for being rotated based on the rotation parameter to the three-dimensional face model, three after being rotated Tie up faceform；

Converting unit, for human face three-dimensional model after the rotation to be converted to two-dimension human face image；

Integrated unit, for merging the two-dimension human face image with the template image.

A kind of storage medium, the storage medium are stored with a plurality of instruction, and described instruction is suitable for processor and is loaded, with Execute the step in any image interfusion method provided by present invention implementation.

The available two dimensional image to be fused of the embodiment of the present invention, and according to the face building three in two dimensional image to be fused Faceform is tieed up, and, the rotation parameter of face in three dimensions in template image and template image is obtained, is then based on Rotation parameter rotates three-dimensional face model, three-dimensional face model after being rotated, then will rotation after three-dimensional face model Two-dimension human face image is converted to, at this point it is possible to two-dimension human face image is merged with template image, such as by two-dimension human face figure Face region of the face fusion into template image as in.The program passes through building and rotated three dimensional faceform, from And two-dimension human face image can be obtained based on three-dimensional face model after rotation, so that face in two-dimension human face image and template image Direction it is consistent, so that the face in two-dimension human face image to be precisely fused to the face region in template image, improve Effect that face in image is merged.

Detailed description of the invention

To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.

Fig. 1 is the schematic diagram of a scenario of image interfusion method provided in an embodiment of the present invention；

Fig. 2 is the flow diagram of image interfusion method provided in an embodiment of the present invention；

Fig. 3 (a) is the schematic diagram of two dimensional image to be fused provided in an embodiment of the present invention；

Fig. 3 (b) is the schematic diagram of template image provided in an embodiment of the present invention；

Fig. 4 (a) is the schematic diagram of three-dimensional face model provided in an embodiment of the present invention；

Fig. 4 (b) is the schematic diagram of three-dimensional face model after rotation provided in an embodiment of the present invention；

Fig. 5 is the schematic diagram of two-dimension human face image provided in an embodiment of the present invention；

Fig. 6 (a) is the schematic diagram that facial feature points detection provided in an embodiment of the present invention obtains human face characteristic point；

Fig. 6 (b) is the schematic diagram that tessellation provided in an embodiment of the present invention obtains face delta-shaped region；

Fig. 7 is the schematic diagram of fused image provided in an embodiment of the present invention；

Fig. 8 is another flow diagram of image interfusion method provided in an embodiment of the present invention；

Fig. 9 is the structural schematic diagram of image fusion device provided in an embodiment of the present invention；

Figure 10 is another structural schematic diagram of image fusion device provided in an embodiment of the present invention；

Figure 11 is the structural schematic diagram of terminal provided in an embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts Example, shall fall within the protection scope of the present invention.

The embodiment of the present invention provides a kind of image interfusion method, device and storage medium.

Referring to Fig. 1, Fig. 1 is the schematic diagram of a scenario of image interfusion method provided by the embodiment of the present invention, wherein image Fusing device, which specifically can integrate, to be had storage element in tablet computer, mobile phone, laptop and desktop computer etc. and installs There is microprocessor and have in the terminal of operational capability, which can be used for receiving the image co-registration instruction of user's input, base Two dimensional image to be fused is obtained in image co-registration instruction, which includes the face of user, for example, can be to clothes Business device sends image acquisition request, and receives to be fused two dimensional image of the server based on return；Or it is acquired by camera The image of user obtains two dimensional image to be fused etc..At this point it is possible to construct three-dimensional people according to the face in two dimensional image to be fused Face model, for example, can to server transmission pattern acquisition request, and receive server returned based on model acquisition request it is pre- If faceform, three-dimensional coordinate of the face in three-dimensional planar in two dimensional image to be fused is obtained according to default faceform Point, and three-dimensional face model etc. is constructed according to three-dimensional coordinate point, it, can be by three-dimensional face model after obtaining three-dimensional face model Server is uploaded to be stored.And available template image, it include face in the template image, such as can be from clothes Template image is obtained on business device, at this point it is possible to obtain the rotation parameter of face in three dimensions in template image.Then, Three-dimensional face model is rotated based on rotation parameter, three-dimensional face model after being rotated, and face after rotation is three-dimensional Model conversion is two-dimension human face image.Finally, two-dimension human face image can be merged with template image, for example, can divide It is other that facial feature points detection is carried out to two-dimension human face image and template image, obtain corresponding human face characteristic point, and according to Face region of the human face characteristic point by the face fusion in two-dimension human face image into template image, schemes after being merged Picture；Etc..

It should be noted that the schematic diagram of a scenario of image interfusion method shown in FIG. 1 is only an example, the present invention is real The scene for applying the image interfusion method of example description is technical solution in order to more clearly illustrate the embodiment of the present invention, not structure In pairs in the restriction of technical solution provided in an embodiment of the present invention, those of ordinary skill in the art are it is found that with image co-registration side The differentiation of method and the appearance of new business scene, technical solution provided in an embodiment of the present invention is for similar technical problem, equally It is applicable in.

It is described in detail separately below.

In the present embodiment, it will be described from the angle of image fusion device, which can specifically collect At tablet computer, mobile phone, laptop and desktop computer etc. have storage element and microprocessor is installed and have fortune In the terminal of calculation ability.

A kind of image interfusion method, including：Two dimensional image to be fused is obtained, and according to the face in two dimensional image to be fused Construct three-dimensional face model；Template image is obtained, and obtains the rotation parameter of face in three dimensions in template image； Three-dimensional face model is rotated based on rotation parameter, three-dimensional face model after being rotated；Face three-dimensional mould after rotating Type is converted to two-dimension human face image；Two-dimension human face image is merged with template image.

Referring to Fig. 2, Fig. 2 is the flow diagram for the image interfusion method that one embodiment of the invention provides.The image melts Conjunction method may include：

In step s101, two dimensional image to be fused is obtained, and three-dimensional is constructed according to the face in two dimensional image to be fused Faceform.

It may include one or more face in the two dimensional image to be fused, can also include other objects, wherein should Face can be positive face (i.e. positive face), or face (i.e. side face) etc. at any angle.For example, as shown in Fig. 3 (a), It may include a positive face in the two dimensional image to be fused.

Two dimensional image to be fused can be image fusion device during taking pictures, the collected figure of camera of taking pictures Picture；Two dimensional image to be fused is also possible to the image that image fusion device prestores, two dimensional image to be fused can also be by Search or the image got from image data base etc. on internet；Certainly, two dimensional image to be fused is also possible to pass through The image that other modes are got, specific acquisition modes are not construed as limiting here.

After obtaining two dimensional image to be fused, image fusion device can be constructed according to the face in two dimensional image to be fused Three-dimensional face model, in some embodiments, according to the step of the face building three-dimensional face model in two dimensional image to be fused Suddenly may include：

Obtain default faceform；According to default faceform, the face obtained in two dimensional image to be fused is flat in three-dimensional Three-dimensional coordinate point in face, and three-dimensional face model is constructed according to three-dimensional coordinate point.

Wherein, default faceform can be three-dimensional average face model, which can be in advance Setting can generate different expression and difference by the location information of the three-dimensional coordinate point of the three-dimensional average face model of change The three-dimensional face model of the user of identity, that is to say, that all three-dimensional face models can be by three-dimensional average face model Three-dimensional coordinate point on increase offset to indicate.

Image fusion device can receive setting instruction, and default faceform is arranged based on setting instruction；Alternatively, image Fusing device can be to server transmission pattern acquisition request, and receives the default people that server is returned based on model acquisition request Face model.After obtaining default faceform, image fusion device can obtain X-Y scheme to be fused according to default faceform Three-dimensional coordinate point of the face in three-dimensional planar as in, and three-dimensional face model is constructed according to three-dimensional coordinate point.

In some embodiments, according to default faceform, the face obtained in two dimensional image to be fused is flat in three-dimensional Three-dimensional coordinate point in face, and the step of constructing three-dimensional face model according to three-dimensional coordinate point may include：

(1) face obtained in two dimensional image to be fused is projected in the first projective parameter in two-dimensional surface；

(2) it obtains default faceform and is projected in the second projective parameter in two-dimensional surface；

(3) according to the first projective parameter and the second projective parameter, the face obtained in two dimensional image to be fused is flat in three-dimensional Three-dimensional coordinate point in face；

(4) three-dimensional face model is constructed according to three-dimensional coordinate point.

Specifically, image fusion device constructs the process of three-dimensional face model according to the face in two dimensional image to be fused In, face can be extracted from two dimensional image to be fused, and obtain the face in two dimensional image to be fused and be projected in two-dimentional put down The first projective parameter in face, first projective parameter can be coordinate points of the face in two-dimensional surface.

And image fusion device obtains default faceform and is projected in the second projective parameter in two-dimensional surface, this Two projective parameters can be coordinate points of the default faceform in two-dimensional surface.Since the three-dimensional average face of change can be passed through The location information of the three-dimensional coordinate point of model generates the three-dimensional face model of the user of different expression and different identity, also It is to say that all three-dimensional face models can be indicated by increasing offset on the three-dimensional coordinate point of three-dimensional average face model, Therefore, the three-dimensional face model of the face of user can be indicated with following formula (1)：

Wherein, the M in formula (1) indicates the three-dimensional face model of face,Indicate that default faceform is (i.e. three-dimensional average Faceform),N is the points for the three-dimensional coordinate point that three-dimensional average face model includes；A_idP_idIndicate that identity is inclined Transposition,A_idFor m_idTie up identity base, P_idFor identification parameters；A_expP_expIndicate expression shift term,A_expFor m_expDimension table feelings base, P_expFor expression parameter.

Obtain the first projective parameter that face is projected in two-dimensional surface and default faceform to be projected in two dimension flat After the second projective parameter in face, image fusion device can obtain face according to the first projective parameter and the second projective parameter Three-dimensional coordinate point in three-dimensional planar, for example, shown in following formula (2) and formula (3)：

arg min||X_3d-X_2d|| (2)

X_3d=p (M_keypoint) (3)

Wherein, X_3dIndicate the three-dimensional coordinate point of default faceform, M_keypointIndicate default faceform in two-dimensional surface The second interior projective parameter, p are projection function, X_2dThe first projective parameter being projected in for face in two-dimensional surface.

Wherein, projection may include rectangular projection and perspective projection etc., can be with by taking rectangular projection carries out three-dimensional modeling as an example Rectangular projection is carried out by following formula (4)：

X_3d=S × R × M_keypoint+T (4)

Wherein, S is scaling coefficient, and R is coefficient of rotary, and T is two-dimension translational component, therefore, can pass through formula (1), public affairs Formula (3) and formula (4) iterative solution formula (2), can be obtained identification parameters P_idWith expression parameter P_exp, scaling coefficient S, rotation Coefficients R and two-dimension translational component T, according to parameter [P_id, P_exp, S, R, T] and three-dimensional coordinate point is obtained, according to three-dimensional coordinate point Generate three-dimensional face model.That is, can be with if the three-dimensional face model of all angles is projected to two-dimensional surface Face matches in two dimensional image to be fused, then this three-dimensional face model is exactly the three-dimensional face model for needing to obtain.Example Such as, the three-dimensional face model such as Fig. 4 (a) constructed according to the face in two dimensional image to be fused is shown.

It should be noted that three-dimensional face model is also other than it can construct three-dimensional face model by above-mentioned formula It can be and obtain by other means, specific building mode is it is not limited here.

It should be noted that when the face in two dimensional image to be fused includes multiple, image fusion device can to A face is extracted arbitrarily in fusion two dimensional image to construct three-dimensional face model, alternatively, image fusion device can export choosing Select item, and receive user selected from two dimensional image to be fused based on options wherein a face construct three-dimensional face mould Type alternatively, image fusion device can extract all faces from two dimensional image to be fused, and distinguishes structure according to each face Build three-dimensional face model；Etc..

In step s 102, template image is obtained, and obtains the rotation of face in three dimensions in template image Parameter.

Wherein, step S102 can be executed before step S101, can also be executed after step slol, specific to execute Sequence can carry out flexible setting according to actual needs, be not construed as limiting here.

It may include one or more face in the template image, can also include other objects, wherein the face can To be positive face (i.e. positive face), or face (i.e. side face) etc. at any angle.For example, as shown in Fig. 3 (b), the template It may include a side face in image.Template image can be image fusion device by camera acquired image of taking pictures, It is also possible to the pre-set image of image fusion device, can also be by searching on the internet or from image data base In the image etc. that gets；Certainly, template image is also possible to get by other means, and specific acquisition modes are here not It limits.

The rotation of face in three dimensions after obtaining template image, in the available template image of image fusion device Turn parameter, which can be the steering in face X-axis in three dimensions, Y-axis and Z-direction, for example, working as mould Face in plate image be positive face when, it is 0 that the rotation parameter of positive face in three dimensions in template image, which can be set,；Work as mould When face in plate image is side face, the side face that can be set in template image is relative to the rotation parameter of positive face：In X-axis side The x angle rotated upwards, the y angle rotated in the Y-axis direction and the z angle rotated in the Z-axis direction.

Image fusion device can receive the rotation parameter of user's input, and set the rotation parameter in template image Face rotation parameter in three dimensions, i.e. the rotation parameter can be by being manually arranged.

In some embodiments, the step of obtaining the face rotation parameter in three dimensions in template image can be with Including：Threedimensional model is constructed according to the face in template image；Based on face in obtaining three-dimensional model template image in three-dimensional space Between in rotation parameter.

In some embodiments, may include according to the step of face building threedimensional model in template image：

The face obtained in template image is projected in the third projective parameter in two-dimensional surface；Default faceform is obtained, And default faceform is projected in the 4th projective parameter in two-dimensional surface；According to third projective parameter and the 4th projective parameter, Obtain target three-dimensional coordinate point of the face in template image in three-dimensional planar；Three-dimensional mould is constructed according to target three-dimensional coordinate point Type.

Specifically, image fusion device can extract face from template image, and obtain the face in template image The third projective parameter being projected in two-dimensional surface, the third projective parameter can be coordinate points of the face in two-dimensional surface. And image fusion device available default faceform (the default faceform and the above-mentioned default faceform referred to Unanimously, as three-dimensional average face model) it is projected in the 4th projective parameter in two-dimensional surface, the 4th projective parameter can be Default coordinate points of the faceform in two-dimensional surface.Then above-mentioned formula (1), formula (2), formula (3) and public affairs be can use Face target three three-dimensional planar in of the formula (4) according to third projective parameter and the 4th projective parameter, in calculation template image Coordinate points are tieed up, and threedimensional model is constructed according to target three-dimensional coordinate point.After obtaining threedimensional model, it can be obtained based on threedimensional model The rotation parameter of face in three dimensions in template image is taken, which is the coefficient of rotary in above-mentioned formula (4) R。

In step s 103, three-dimensional face model is rotated based on rotation parameter, three-dimensional face mould after being rotated Type.

After obtaining three-dimensional face model and rotation parameter, rotation parameter can be applied to three-dimensional people by image fusion device On face model, control three-dimensional face model carries out the rotation on the directions such as X-axis, Y-axis and Z axis in three dimensions, so that three-dimensional The direction (including posture and angle etc.) of face is consistent with face direction in Prototype drawing in faceform.For example, based on rotation ginseng Several pairs of three-dimensional face models rotate, after being rotated shown in three-dimensional face model such as Fig. 4 (b).

The face for example, face in the three-dimensional face model is positive, and when the face in template image is side face, in order to enable The direction of face is consistent with face direction in Prototype drawing in three-dimensional face model, can be based on side face in Prototype drawing in three-dimensional space In rotation parameter relative to positive face, the positive face in three-dimensional face model is rotated, three-dimensional face model after being rotated In face be side face.

In another example when the face in three-dimensional face model be side face, and the face in template image be positive face when, in order to make The direction for obtaining face in three-dimensional face model is consistent with face direction in Prototype drawing, can be based on face positive in Prototype drawing in three-dimensional space Between in rotation parameter relative to side face, the side face in three-dimensional face model is rotated, three-dimensional face mould after being rotated Face in type is positive face.

It should be noted that image fusion device can be to after the rotation three after being rotated after three-dimensional face model Dimension faceform pre-processes, such as by brightness adjustment approach, so that the light after rotation in three-dimensional face model on face Quite according to environment holding and Prototype drawing etc..

In step S104, human face three-dimensional model after rotation is converted into two-dimension human face image.

After being rotated after human face three-dimensional model, image fusion device can be converted to human face three-dimensional model after rotation Two-dimension human face image, the two-dimension human face image can be as shown in figure 5, for example, can receive screenshot instruction, directly according to screenshot instruction The image that human face three-dimensional model is shown in display interface after interception rotates is connect, two-dimension human face image is obtained；Alternatively, according to above-mentioned Inverse mapping is carried out according to the method that face constructs three-dimensional face model, is two-dimension human face by human face three-dimensional model inverse mapping after rotation Image, it is, of course, also possible to which human face three-dimensional model after rotation is converted to two-dimension human face image by other means.

In step s105, two-dimension human face image is merged with template image.

Image fusion device can first do facial feature points detection, then divide face delta according to human face characteristic point Then affine transformation is done according to face delta-shaped region in domain, finally by each piece of face three in two-dimension human face image on face Angular domain is fused in template image on the corresponding position of face, can also do the post-processing operations such as some edges are soft Complete facial image fusion.

It should be noted that when the face in two-dimension human face image and template image is multiple, image fusion device Three-dimensional face model can be constructed respectively according to each face in two dimensional image to be fused, and obtained each in template image The rotation parameter of a face in three dimensions respectively rotates each three-dimensional face model based on each rotation parameter, Three-dimensional face model after being rotated；Then human face three-dimensional model after each rotation is converted into two-dimension human face image, and by two Each face is merged with face each in template image in dimension facial image.For example, including the in the two-dimension human face image One face and the second face can be according to the first faces and the second people when including third face and four faces in template image Face constructs the rotation parameter of three-dimensional face model and third face and the 4th face in three dimensions respectively, is based on third The corresponding rotation parameter of face rotates the corresponding three-dimensional face model of the first face, is based on the corresponding rotation of the 4th face Parameter rotates the corresponding three-dimensional face model of the second face, three-dimensional face model after being rotated；Then by each rotation Human face three-dimensional model is converted to two-dimension human face image after turning, finally will be in the first face and template image in two-dimension human face image Third face merged, the second face in two-dimension human face image is merged with the 4th face in template image.

In some embodiments, the step of two-dimension human face image being merged with template image may include：

(1) facial feature points detection is carried out to two-dimension human face image, obtains the first human face characteristic point；

(2) facial feature points detection is carried out to template image, obtains the second human face characteristic point；

(3) according to the first human face characteristic point and the second human face characteristic point, by the face fusion in two-dimension human face image to mould Face region in plate image.

Image fusion device can be by face recognition technology, to eyes, nose, the eyebrow on face in two-dimension human face image Hair and each human face such as mouth carry out characteristic point detection, generate the first human face characteristic point, which can be with It is the location coordinate information of the corresponding key point of each human face, which can be in the exterior contour of face With the edge of each human face or center etc., the number of first human face characteristic point can be 106,83 or 68 etc., Specific number can carry out flexible setting according to actual needs.

And image fusion device can by face recognition technology, on face in template image eyes, nose, Each human face such as eyebrow and mouth carries out characteristic point detection, generates the second human face characteristic point, which can To be the location coordinate information of the corresponding key point of each human face, which can be in the outer wheels of face Edge or center of wide and each human face etc., the number of second human face characteristic point can be 106,83 or 68 Deng specific number can carry out flexible setting according to actual needs.In order to which two-dimension human face image and template image are carried out Precisely fusion, the number that the first human face characteristic point and the second human face characteristic point can be set are consistent.For example, carrying out human face characteristic point The human face characteristic point facial image for being included is detected, as shown in Fig. 6 (a).

At this point, image fusion device can be according to the first human face characteristic point and the second human face characteristic point, by two-dimension human face figure Face region of the face fusion into template image as in.

It in some embodiments, will be in two-dimension human face image according to the first human face characteristic point and the second human face characteristic point Face region of the face fusion into template image the step of include：

(a) according to the first human face characteristic point, tessellation is carried out to the face in two-dimension human face image, is obtained the first Face delta-shaped region；

(b) according to the second human face characteristic point, tessellation is carried out to the face in template image, obtains the second face three Angular domain；

(c) affine transformation is carried out based on the first face delta-shaped region and the second face delta-shaped region, obtains target person Face delta-shaped region；

(d) target face delta-shaped region is fused to the face region in template image.

Specifically, image fusion device can carry out three to the face in two-dimension human face image according to the first human face characteristic point Angular division obtains the first face delta-shaped region, for example, each human face characteristic point is subjected to line according to preset algorithm, from And face can be divided into multiple delta-shaped regions, which can carry out flexible setting according to actual needs.And Tessellation is carried out to the face in template image according to the second human face characteristic point, obtains the second face delta-shaped region, example Such as, tessellation is carried out to face according to human face characteristic point and obtains the facial image comprising face delta-shaped region, such as Fig. 6 (b) shown in.

Then, image fusion device can be based on the corresponding first face delta-shaped region of two-dimension human face image and Prototype drawing As corresponding second face delta-shaped region progress affine transformation, target face delta-shaped region is obtained.For example, can be by first The face delta-shaped region of corresponding position is compared in face delta-shaped region and the second face delta-shaped region, if the first Shapes and sizes of face delta-shaped region of corresponding position etc. are different in face delta-shaped region and the second face delta-shaped region It causes, the face delta-shaped region of the position in the first face delta-shaped region can be zoomed in and out, rotated or be translated at this time Operation, obtains target face delta-shaped region, so that corresponding in target face delta-shaped region and the second face delta-shaped region Face delta-shaped region shapes and sizes of position etc. are consistent.

At this point it is possible to which target face delta-shaped region is fused to the face region in template image, merged Image afterwards, for example, the fused image can be as shown in Figure 7.In order to enable syncretizing effect is more naturally, can be used alpha- The blending algorithms such as blend seamless clone are merged.

It will be illustrated below, when the face in two dimensional image to be fused is positive face, and the face in template image When for side face, three-dimensional face model is constructed according to the positive face in two dimensional image to be fused, in the obtained three-dimensional face model The side face that face is positive in face, and acquisition template image in three dimensions join by the rotation parameter relative to positive face, the rotation Number is the rotation parameter relative to positive face of side face in three dimensions in template image.

At this point, being rotated based on rotation parameter to three-dimensional face model, after being rotated the step of three-dimensional face model May include：Positive face in three-dimensional face model is rotated based on rotation parameter, three-dimensional face model after being rotated, rotation Face after turning in three-dimensional face model is side face.The step of two-dimension human face image is merged with template image can wrap It includes：Face on rear side of rotation in two-dimension human face image is fused to the side face region in template image.To pass through building three The mode of faceform is tieed up to solve the problems, such as that side face fusion is difficult, user only needs normal photographing face image, passes through face image Three-dimensional face model is constructed, then automates rotated three dimensional faceform to consistent with side face direction in Prototype drawing, it finally will rotation Three-dimensional face model is converted to two-dimension human face image afterwards, carries out progress side face between two-dimension human face image and template image and merges. Such as, it is only necessary to face is positive face in two dimensional image to be fused, can be achieved with carrying out side face with the Prototype drawing of each side face merging, Convenient and efficient and effect is fine.

It should be noted that image fusion device can first obtain two dimensional image to be fused, and according to X-Y scheme to be fused Face as in constructs three-dimensional face model；Template image is obtained, threedimensional model is constructed according to the face in template image, and Based on the rotation parameter of face in three dimensions in obtaining three-dimensional model template image；Then, based on rotation parameter to three-dimensional Faceform rotates, three-dimensional face model after being rotated；Secondly, by the human face region in three-dimensional face model after rotation It is merged with the human face region in threedimensional model, three-dimensional face model after being merged, three-dimensional face mould after finally merging Type is converted to two-dimension human face image；Etc..

From the foregoing, it will be observed that the available two dimensional image to be fused of the embodiment of the present invention, and according in two dimensional image to be fused Face constructs three-dimensional face model, and, obtain the rotation ginseng of face in three dimensions in template image and template image Number, be then based on rotation parameter and three-dimensional face model rotated, three-dimensional face model after rotate, then will rotate after three Dimension faceform is converted to two-dimension human face image, at this point it is possible to two-dimension human face image be merged with template image, such as will Face region of the face fusion into template image in two-dimension human face image.The program passes through building and rotated three dimensional people Face model, so as to obtain two-dimension human face image based on three-dimensional face model after rotation, so that two-dimension human face image and template The direction of face is consistent in image, so as to the face place being precisely fused to the face in two-dimension human face image in template image Region improves the effect merged to face in image.

Citing, is described in further detail by the method according to described in above-described embodiment below.

The present embodiment is positive face, and mould so that image fusion device is terminal as an example with the face in two dimensional image to be fused Face in plate image is side face, and the face in two dimensional image and template image to be fused is for one, it is possible to understand that It is that the present embodiment should not be understood as only to facilitate the lifted example of description to two dimensional image to be fused and template image In face direction and number be defined, but regardless of which kind of the face direction in two dimensional image to be fused and template image is Direction and number be it is how many, the process of image co-registration be all it is similar, can be understood according to the example.

Referring to Fig. 8, Fig. 8 is the flow diagram of image interfusion method provided in an embodiment of the present invention.This method process May include：

S201, terminal obtain two dimensional image and template image to be fused.

It wherein, may include a positive face in the two dimensional image to be fused, as shown in Fig. 3 (a)；It can be in the template image It can also include other objects in two dimensional image and template image to be fused certainly as shown in Fig. 3 (b) comprising a side face Body.

Two dimensional image and template image to be fused can be terminal during taking pictures, the collected figure of camera of taking pictures Picture is also possible to the pre-set image of terminal, can also obtain by searching on the internet or from image data base Obtained image etc.；It is of course also possible to be the image got by other means, specific acquisition modes do not limit here It is fixed.For example, two dimensional image to be fused, which can be user's self-timer, obtains the image comprising positive face, template image, which can be, to be preset Image.

S202, terminal obtain three of the face in two dimensional image to be fused in three-dimensional planar according to default faceform Coordinate points are tieed up, and three-dimensional face model is constructed according to three-dimensional coordinate point.

Terminal can obtain three-dimensional of the positive face in three-dimensional planar in two dimensional image to be fused according to default faceform Coordinate points, and three-dimensional face model is constructed according to three-dimensional coordinate point.For example, terminal can according to above-mentioned formula (1), formula (2), Formula (3) and formula (4) obtain the first projective parameter that the face in two dimensional image to be fused is projected in two-dimensional surface, with And obtain default faceform and be projected in the second projective parameter in two-dimensional surface, then thrown according to the first projective parameter and second Shadow parameter obtains three-dimensional coordinate point of the face in two dimensional image to be fused in three-dimensional planar, and according to three-dimensional coordinate point structure Build three-dimensional face model；Etc., which can be as shown in Fig. 4 (a).

S203, terminal construct threedimensional model according to the face in template image, based in obtaining three-dimensional model template image The rotation parameter of face in three dimensions.

Wherein, S203 can be executed before step S202, can also be executed after step S202.

Terminal can obtain the face in template image according to above-mentioned formula (1), formula (2), formula (3) and formula (4) The third projective parameter being projected in two-dimensional surface；It obtains default faceform and default faceform is projected in two-dimensional surface The 4th interior projective parameter；Then according to third projective parameter and the 4th projective parameter, the face in template image is obtained three Target three-dimensional coordinate point in dimensional plane, and threedimensional model is constructed according to target three-dimensional coordinate point；Etc..Obtaining threedimensional model It afterwards, can be based on the rotation parameter relative to positive face in three dimensions of side face in obtaining three-dimensional model template image, the rotation Parameter is the coefficient of rotary R in above-mentioned formula (4).For example, rotation parameter may include the x angle rotated in the X-axis direction, The y angle rotated in the Y-axis direction and the z angle rotated in the Z-axis direction etc..The rotation parameter can also be to be set by user Fixed.

S204, terminal are based on rotation parameter and rotate to three-dimensional face model, three-dimensional face model after being rotated.

Rotation parameter can be applied on three-dimensional face model by terminal, control three-dimensional face model in three dimensions into Rotation on the directions such as row X-axis, Y-axis and Z axis, so that face direction in the direction of face and Prototype drawing in three-dimensional face model Unanimously.For example, can be based on the rotation parameter relative to positive face in three dimensions of side face in Prototype drawing, to three-dimensional face model In positive face rotated, the face after being rotated in three-dimensional face model be side face, three-dimensional face model such as Fig. 4 after rotation (b) shown in.

Human face three-dimensional model after rotation is converted to two-dimension human face image by S205, terminal.

Terminal can receive screenshot instruction, according to human face three-dimensional model after screenshot instruction directly interception rotation in display interface The image of interior display, obtains two-dimension human face image；Alternatively, being carried out according to the above-mentioned method for constructing three-dimensional face model according to face Human face three-dimensional model inverse mapping after rotation is two-dimension human face image by inverse mapping, it is, of course, also possible to by other means will rotation Human face three-dimensional model is converted to two-dimension human face image afterwards.For example, the two-dimension human face image can be as shown in figure 5, obtained two dimension Face in facial image is side face.

S206, terminal carry out facial feature points detection to two-dimension human face image and template image respectively, and correspondence obtains first Human face characteristic point and the second human face characteristic point.

Terminal can by face recognition technology respectively on face in two-dimension human face image eyes, nose, eyebrow and Each human face such as mouth carries out characteristic point detection, generates the first human face characteristic point, and respectively to face in template image On each human face such as eyes, nose, eyebrow and mouth carry out characteristic point detection, generate the second human face characteristic point.This One human face characteristic point and the second human face characteristic point can be the location coordinate information of the corresponding key point of each human face, this One human face characteristic point and the second human face characteristic point can be in the exterior contours of face and the edge of each human face or center etc.. The number of first human face characteristic point and the second human face characteristic point can be 106,83 or 68 etc., and specific number can root Flexible setting is carried out according to actual needs.For example, carrying out the human face characteristic point facial image that facial feature points detection is included, such as Shown in Fig. 6 (a).

S207, terminal melt the face in two-dimension human face image according to the first human face characteristic point and the second human face characteristic point It is bonded to the face region in template image.

Terminal can carry out tessellation to the face in two-dimension human face image according to the first human face characteristic point, obtain the One face delta-shaped region, and, tessellation is carried out to the face in template image according to the second human face characteristic point, is obtained Second face delta-shaped region obtains for example, carrying out tessellation to face according to human face characteristic point comprising face triangle The facial image in region, as shown in Fig. 6 (b).

Then, terminal can be corresponding based on the corresponding first face delta-shaped region of two-dimension human face image and template image Second face delta-shaped region carries out affine transformation, obtains target face delta-shaped region.For example, face in two-dimension human face image Line between characteristic point 56,62 and 64 constitutes a face delta-shaped region A1, human face characteristic point 56,62 and in template image Line between 64 constitutes a face delta-shaped region A2, and face delta-shaped region A1 and face delta-shaped region A2 is carried out Compare, it, at this time can be to face if face delta-shaped region A1 and the shapes and sizes of face delta-shaped region A2 etc. are inconsistent Delta-shaped region A1 such as zooms in and out, rotates or translates at the operation, obtains target face delta-shaped region A, the target face triangle Shape region A can be consistent with face delta-shaped region A2 shapes and sizes etc., other face delta-shaped regions also carry out similarly Processing.At this point, target face delta-shaped region can be fused to the face region in template image by terminal, merged Image afterwards, for example, the fused image can be as shown in Figure 7.

The embodiment of the present invention constructs three-dimensional face model by the positive face in two dimensional image to be fused, and according to template image In the rotation parameter of side face automate rotated three dimensional faceform so that side face in positive face and Prototype drawing in three-dimensional face model Direction is consistent, and three-dimensional face model after rotation is finally converted to two-dimension human face image, by two-dimension human face image and template image Carry out side face fusion.The program improves two-dimentional side face syncretizing effect by way of constructing three-dimensional face model, it is only necessary to wait melt It closes face in two dimensional image to be positive face, can be achieved with and side face is perfectly combined at any angle in Prototype drawing, merged The syncretizing effect of image is very good afterwards, and fused face is naturally beautiful, and whole-course automation, convenient and efficient.For example, can be with The application scenarios for improving mad Face Changing, increase entertaining and playability.

For convenient for better implementation image interfusion method provided in an embodiment of the present invention, the embodiment of the present invention also provides one kind Device based on above-mentioned image interfusion method.Wherein the meaning of noun is identical with above-mentioned image interfusion method, and specific implementation is thin Section can be with reference to the explanation in embodiment of the method.

Referring to Fig. 9, Fig. 9 is the structural schematic diagram of image fusion device provided in an embodiment of the present invention, the wherein image Fusing device may include construction unit 301, acquiring unit 302, rotary unit 303, converting unit 304 and integrated unit 305 Deng.

Wherein, construction unit 301, for obtaining two dimensional image to be fused, and according to the face in two dimensional image to be fused Construct three-dimensional face model.

It may include one or more face in the two dimensional image to be fused, can also include other objects, wherein should Face can positive face or side face etc..For example, may include a positive face in the two dimensional image to be fused as shown in Fig. 3 (a).

Two dimensional image to be fused can be during taking pictures, camera acquired image of taking pictures；It is also possible to figure The image prestored as fusing device；It can also be the image by searching for or getting on the internet from image data base Deng；Certainly, two dimensional image to be fused is also possible to the image got by other means, and specific acquisition modes are not made here It limits.

In some embodiments, construction unit 301 may include the first acquisition subelement and the first building subelement etc. It specifically can be as follows：

First obtains subelement, for obtaining default faceform；

First building subelement, for obtaining the face in two dimensional image to be fused in three-dimensional according to faceform is preset Three-dimensional coordinate point in plane, and three-dimensional face model is constructed according to three-dimensional coordinate point.

First acquisition subelement can receive setting instruction, and default faceform is arranged based on setting instruction；Alternatively, the One obtain subelement can to server transmission pattern acquisition request, and receive server returned based on model acquisition request it is pre- If faceform.After obtaining default faceform, the first building subelement can obtain to be fused according to default faceform Three-dimensional coordinate point of the face in three-dimensional planar in two dimensional image, and three-dimensional face model is constructed according to three-dimensional coordinate point.

In some embodiments, the first building subelement specifically can be used for：Obtain the people in two dimensional image to be fused Face is projected in the first projective parameter in two-dimensional surface；Obtain the second projection ginseng that default faceform is projected in two-dimensional surface Number；According to the first projective parameter and the second projective parameter, three of the face in two dimensional image to be fused in three-dimensional planar are obtained Tie up coordinate points；Three-dimensional face model is constructed according to three-dimensional coordinate point.

Specifically, the first building subelement constructs the process of three-dimensional face model according to the face in two dimensional image to be fused In, face can be extracted from two dimensional image to be fused, and obtain the face in two dimensional image to be fused and be projected in two-dimentional put down The first projective parameter in face, first projective parameter can be coordinate points of the face in two-dimensional surface.And first building The available default faceform of subelement is projected in the second projective parameter in two-dimensional surface, which can be Default coordinate points of the faceform in two-dimensional surface.Since the three-dimensional coordinate point of the three-dimensional average face model of change can be passed through Location information therefore can pass through above-mentioned public affairs to generate the three-dimensional face model of the user of different expression and different identity Formula (1), formula (2), formula (3) and formula (4) obtain two dimension to be fused according to the first projective parameter and the second projective parameter Three-dimensional coordinate point of the face in three-dimensional planar in image, and three-dimensional face model is constructed according to three-dimensional coordinate point.Namely It says, it can be with face phase in two dimensional image to be fused if the three-dimensional face model of all angles is projected to two-dimensional surface Match, then this three-dimensional face model is exactly the three-dimensional face model for needing to obtain.For example, according in two dimensional image to be fused Shown in the three-dimensional face model that face constructs such as Fig. 4 (a).

It should be noted that when the face in two dimensional image to be fused includes multiple, the first building subelement can be from A face is extracted in two dimensional image to be fused arbitrarily to construct three-dimensional face model, alternatively, the first building subelement can be defeated Options out, and receive user selected from two dimensional image to be fused based on options wherein a face construct three-dimensional face Model, alternatively, the first building subelement can extract all faces from two dimensional image to be fused, and according to each face point It Gou Jian not three-dimensional face model；Etc..

Acquiring unit 302, for obtaining the rotation of the face in template image, and acquisition template image in three dimensions Turn parameter.

It may include one or more face in the template image, can also include other objects, wherein the face can To be positive face or side face at any angle.For example, may include a side face in the template image as shown in Fig. 3 (b).Mould Plate image can be through camera acquired image of taking pictures, and be also possible to pre-set image, can also be by Search or the image got from image data base etc. on internet；Certainly, template image is also possible to through its other party Formula is got, and specific acquisition modes are not construed as limiting here.

The rotation parameter of face in three dimensions in the available template image of acquiring unit 302, the rotation parameter It can be the steering in face X-axis in three dimensions, Y-axis and Z-direction, for example, the face in template image is positive When face, it is 0 that the rotation parameter of positive face in three dimensions in template image, which can be set,；When the face in template image is side When face, the side face that can be set in template image is relative to the rotation parameter of positive face：The x angle that rotates in the X-axis direction, The y angle rotated in Y direction and the z angle rotated in the Z-axis direction.

Acquiring unit 302 can receive the rotation parameter of user's input, and set the rotation parameter in template image Face rotation parameter in three dimensions, i.e. the rotation parameter can be by being manually arranged.

In some embodiments, acquiring unit 302 may include that the second building subelement and second obtain subelement etc., It specifically can be as follows：

Second building subelement, for constructing threedimensional model according to the face in template image；

Second obtains subelement, for the rotation ginseng based on face in obtaining three-dimensional model template image in three dimensions Number.

In some embodiments, the second building subelement specifically can be used for：

The face obtained in template image is projected in the third projective parameter in two-dimensional surface；

It obtains default faceform and default faceform is projected in the 4th projective parameter in two-dimensional surface；

According to third projective parameter and the 4th projective parameter, target of the face in template image in three-dimensional planar is obtained Three-dimensional coordinate point；

Threedimensional model is constructed according to target three-dimensional coordinate point.

Specifically, the second building subelement can extract face from template image, and obtain the people in template image Face is projected in the third projective parameter in two-dimensional surface, which can be coordinate of the face in two-dimensional surface Point.And the available default faceform of image fusion device is projected in the 4th projective parameter in two-dimensional surface, the 4th Projective parameter can be coordinate points of the default faceform in two-dimensional surface.Then above-mentioned formula (1), formula be can use (2), formula (3) and formula (4) are according to third projective parameter and the 4th projective parameter, and the face in calculation template image is in three-dimensional Target three-dimensional coordinate point in plane, and threedimensional model is constructed according to target three-dimensional coordinate point.After obtaining threedimensional model, second Obtaining subelement can be based on the rotation parameter of face in three dimensions, the rotation parameter in obtaining three-dimensional model template image Coefficient of rotary R as in above-mentioned formula (4).

Rotary unit 303, for being rotated based on rotation parameter to three-dimensional face model, three-dimensional face after being rotated Model.

After obtaining three-dimensional face model and rotation parameter, rotation parameter can be applied to three-dimensional people by rotary unit 303 On face model, control three-dimensional face model carries out the rotation on the directions such as X-axis, Y-axis and Z axis in three dimensions, so that three-dimensional The direction of face is consistent with face direction in Prototype drawing in faceform.For example, based on rotation parameter to three-dimensional face model into Row rotation, after being rotated shown in three-dimensional face model such as Fig. 4 (b).

Converting unit 304 is converted to two-dimension human face image for human face three-dimensional model after rotating.

After being rotated after human face three-dimensional model, converting unit 304 can be converted to human face three-dimensional model after rotation Two-dimension human face image, the two-dimension human face image can be as shown in figure 5, for example, can receive screenshot instruction, directly according to screenshot instruction The image that human face three-dimensional model is shown in display interface after interception rotates is connect, two-dimension human face image is obtained；Alternatively, according to above-mentioned Inverse mapping is carried out according to the method that face constructs three-dimensional face model, is two-dimension human face by human face three-dimensional model inverse mapping after rotation Image, it is, of course, also possible to which human face three-dimensional model after rotation is converted to two-dimension human face image by other means.

Integrated unit 305, for merging two-dimension human face image with template image.

Integrated unit 305 can first do facial feature points detection, then divide face delta according to human face characteristic point Then affine transformation is done according to face delta-shaped region in domain, finally by each piece of face three in two-dimension human face image on face Angular domain is fused in template image on the corresponding position of face, can also do the post-processing operations such as some edges are soft Complete facial image fusion.

It should be noted that integrated unit 305 can when the face in two-dimension human face image and template image is multiple To construct three-dimensional face model respectively according to each face in two dimensional image to be fused, and obtain each in template image The rotation parameter of face in three dimensions respectively rotates each three-dimensional face model based on each rotation parameter, obtains Three-dimensional face model after to rotation；Then human face three-dimensional model after each rotation is converted into two-dimension human face image, and will be two-dimentional Each face is merged with face each in template image in facial image.For example, when including first in two-dimension human face image Face and the second face can be according to the first faces and the second face when including third face and four faces in template image The rotation parameter of three-dimensional face model and third face and the 4th face in three dimensions is constructed respectively, is based on the third party The corresponding rotation parameter of face rotates the corresponding three-dimensional face model of the first face, based on the corresponding rotation ginseng of the 4th face It is several that the corresponding three-dimensional face model of second face is rotated, three-dimensional face model after being rotated；Then by each rotation Human face three-dimensional model is converted to two-dimension human face image afterwards, finally will be in the first face and template image in two-dimension human face image Third face is merged, and the second face in two-dimension human face image is merged with the 4th face in template image.

In some embodiments, as shown in Figure 10, integrated unit 305 may include the first detection sub-unit 3051, Two detection sub-units 3052 and fusion subelement 3053 etc., specifically can be as follows：

First detection sub-unit 3051 obtains the first face for carrying out facial feature points detection to two-dimension human face image Characteristic point；

Second detection sub-unit 3052 obtains the second face characteristic for carrying out facial feature points detection to template image Point；

Subelement 3053 is merged, is used for according to the first human face characteristic point and the second human face characteristic point, by two-dimension human face image In face region of the face fusion into template image.

First detection sub-unit 3051 can by face recognition technology, on face in two-dimension human face image eyes, Each human face such as nose, eyebrow and mouth carries out characteristic point detection, generates the first human face characteristic point, first face characteristic Point can be the location coordinate information of the corresponding key point of each human face, which can be in the outer of face The edge or center etc. of contouring and each human face, the number of first human face characteristic point can be 106,83 or 68 A etc., specific number can carry out flexible setting according to actual needs.

Second detection sub-unit 3052 can by face recognition technology, on face in template image eyes, nose, Each human face such as eyebrow and mouth carries out characteristic point detection, generates the second human face characteristic point, which can To be the location coordinate information of the corresponding key point of each human face, which can be in the outer wheels of face Edge or center of wide and each human face etc., the number of second human face characteristic point can be 106,83 or 68 Deng specific number can carry out flexible setting according to actual needs.In order to which two-dimension human face image and template image are carried out Precisely fusion, the number that the first human face characteristic point and the second human face characteristic point can be set are consistent.For example, carrying out human face characteristic point The human face characteristic point facial image for being included is detected, as shown in Fig. 6 (a).

At this point, fusion subelement 3053 can be according to the first human face characteristic point and the second human face characteristic point, by two-dimension human face Face region of the face fusion into template image in image.

In some embodiments, fusion subelement 3053 specifically can be used for：

According to the first human face characteristic point, tessellation is carried out to the face in two-dimension human face image, obtains the first face Delta-shaped region；According to the second human face characteristic point, tessellation is carried out to the face in template image, obtains the second face three Angular domain；Affine transformation is carried out based on the first face delta-shaped region and the second face delta-shaped region, obtains target face Delta-shaped region；Target face delta-shaped region is fused to the face region in template image.

Specifically, fusion subelement 3053 can carry out the face in two-dimension human face image according to the first human face characteristic point Tessellation obtains the first face delta-shaped region, for example, each human face characteristic point is subjected to line according to preset algorithm, So as to which face is divided into multiple delta-shaped regions, which can carry out flexible setting according to actual needs.With And tessellation is carried out to the face in template image according to the second human face characteristic point, the second face delta-shaped region is obtained, For example, carrying out tessellation to face according to human face characteristic point obtains the facial image comprising face delta-shaped region, such as Fig. 6 (b) shown in.

Then, fusion subelement 3053 can be based on the corresponding first face delta-shaped region of two-dimension human face image and template The corresponding second face delta-shaped region of image carries out affine transformation, obtains target face delta-shaped region.For example, can be by The face delta-shaped region of corresponding position is compared in one face delta-shaped region and the second face delta-shaped region, if first Shapes and sizes of face delta-shaped region of corresponding position etc. are not no in face delta-shaped region and the second face delta-shaped region Unanimously, the face delta-shaped region of the position in the first face delta-shaped region can be zoomed in and out, rotated or is translated at this time Deng operation, target face delta-shaped region is obtained, so that right in target face delta-shaped region and the second face delta-shaped region Answer face delta-shaped region shapes and sizes of position etc. consistent.At this point it is possible to which target face delta-shaped region is fused to mould Face region in plate image obtains fused image, such as the fused image can be as shown in Figure 7.

At this point, rotary unit 303 specifically can be used for：Positive face in three-dimensional face model is revolved based on rotation parameter Turn, three-dimensional face model after being rotated, the face after rotation in three-dimensional face model is side face.

Integrated unit 305 specifically can be used for：Face on rear side of rotation in two-dimension human face image is fused in template image Side face region.To solve the problems, such as that side face fusion is difficult by way of constructing three-dimensional face model, user is only needed Normal photographing face image constructs three-dimensional face model by face image, then automates rotated three dimensional faceform and arrive and mould Side face direction is consistent in plate figure, and three-dimensional face model after rotation is finally converted to two-dimension human face image, carries out two-dimension human face figure As being merged with progress side face between template image.Such as, it is only necessary to face is positive face in two dimensional image to be fused, can be achieved with The Prototype drawing of each side face carries out side face fusion, and convenient and efficient and effect is fine.

From the foregoing, it will be observed that the embodiment of the present invention can obtain two dimensional image to be fused by construction unit 301, and according to be fused Face in two dimensional image constructs three-dimensional face model, and, it is obtained in template image and template image by acquiring unit 302 Then the rotation parameter of face in three dimensions is based on rotation parameter by rotary unit 303 and revolves to three-dimensional face model Turn, three-dimensional face model after being rotated, then three-dimensional face model after rotation is converted into two-dimension human face figure by converting unit 304 Picture, at this point, integrated unit 305 can merge two-dimension human face image with template image, such as will be in two-dimension human face image Face region of the face fusion into template image.The program passes through building and rotated three dimensional faceform, so as to To obtain two-dimension human face image based on three-dimensional face model after rotation, so that the side of two-dimension human face image and face in template image To consistent, so that the face in two-dimension human face image to be precisely fused to the face region in template image, improve pair The effect that face is merged in image.

Correspondingly, the embodiment of the present invention also provides a kind of terminal, as shown in figure 11, the terminal may include radio frequency (RF, Radio Frequency) circuit 601, the memory 602, defeated that includes one or more computer readable storage medium Enter unit 603, display unit 604, sensor 605, voicefrequency circuit 606, Wireless Fidelity (WiFi, Wireless Fidelity) The components such as module 607, the processor 608 for including one or more than one processing core and power supply 609.This field skill Art personnel are appreciated that the restriction of the not structure paired terminal of terminal structure shown in Figure 11, may include than illustrate it is more or Less component perhaps combines certain components or different component layouts.Wherein：

RF circuit 601 can be used for receiving and sending messages or communication process in, signal sends and receivees, particularly, by base station After downlink information receives, one or the processing of more than one processor 608 are transferred to；In addition, the data for being related to uplink are sent to Base station.In general, RF circuit 601 includes but is not limited to antenna, at least one amplifier, tuner, one or more oscillators, uses Family identity module (SIM, Subscriber Identity Module) card, transceiver, coupler, low-noise amplifier (LNA, Low Noise Amplifier), duplexer etc..In addition, RF circuit 601 can also by wireless communication with network and its He communicates equipment.Any communication standard or agreement, including but not limited to global system for mobile telecommunications system can be used in the wireless communication Unite (GSM, Global System of Mobile communication), general packet radio service (GPRS, General Packet Radio Service), CDMA (CDMA, Code Division Multiple Access), wideband code division it is more Location (WCDMA, Wideband Code Division Multiple Access), long term evolution (LTE, Long Term Evolution), Email, short message service (SMS, Short Messaging Service) etc..

Memory 602 can be used for storing software program and module, and processor 608 is stored in memory 602 by operation Software program and module, thereby executing various function application and data processing.Memory 602 can mainly include storage journey Sequence area and storage data area, wherein storing program area can the (ratio of application program needed for storage program area, at least one function Such as sound-playing function, image player function) etc.；Storage data area, which can be stored, uses created data according to terminal (such as audio data, phone directory etc.) etc..In addition, memory 602 may include high-speed random access memory, can also include Nonvolatile memory, for example, at least a disk memory, flush memory device or other volatile solid-state parts.Phase Ying Di, memory 602 can also include Memory Controller, to provide processor 608 and input unit 603 to memory 602 Access.

Input unit 603 can be used for receiving the number or character information of input, and generate and user setting and function Control related keyboard, mouse, operating stick, optics or trackball signal input.Specifically, in a specific embodiment In, input unit 603 may include touch sensitive surface and other input equipments.Touch sensitive surface, also referred to as touch display screen or touching Control plate, collect user on it or nearby touch operation (such as user using any suitable object such as finger, stylus or Operation of the attachment on touch sensitive surface or near touch sensitive surface), and corresponding connection dress is driven according to preset formula It sets.Optionally, touch sensitive surface may include both touch detecting apparatus and touch controller.Wherein, touch detecting apparatus is examined The touch orientation of user is surveyed, and detects touch operation bring signal, transmits a signal to touch controller；Touch controller from Touch information is received on touch detecting apparatus, and is converted into contact coordinate, then gives processor 608, and can reception processing Order that device 608 is sent simultaneously is executed.Furthermore, it is possible to a variety of using resistance-type, condenser type, infrared ray and surface acoustic wave etc. Type realizes touch sensitive surface.In addition to touch sensitive surface, input unit 603 can also include other input equipments.Specifically, other are defeated Entering equipment can include but is not limited to physical keyboard, function key (such as volume control button, switch key etc.), trace ball, mouse One of mark, operating stick etc. are a variety of.

Display unit 604 can be used for showing information input by user or be supplied to user information and terminal it is various Graphical user interface, these graphical user interface can be made of figure, text, icon, video and any combination thereof.Display Unit 604 may include display panel, optionally, can using liquid crystal display (LCD, Liquid Crystal Display), The forms such as Organic Light Emitting Diode (OLED, Organic Light-Emitting Diode) configure display panel.Further , touch sensitive surface can cover display panel, after touch sensitive surface detects touch operation on it or nearby, send processing to Device 608 is followed by subsequent processing device 608 and is provided on a display panel accordingly according to the type of touch event to determine the type of touch event Visual output.Although touch sensitive surface and display panel are to realize input and defeated as two independent components in Figure 11 Enter function, but in some embodiments it is possible to touch sensitive surface and display panel is integrated and realizes and outputs and inputs function.

Terminal may also include at least one sensor 605, such as optical sensor, motion sensor and other sensors. Specifically, optical sensor may include ambient light sensor and proximity sensor, wherein ambient light sensor can be according to ambient light Light and shade adjust the brightness of display panel, proximity sensor can close display panel and/or back when terminal is moved in one's ear Light.As a kind of motion sensor, gravity accelerometer can detect (generally three axis) acceleration in all directions Size can detect that size and the direction of gravity when static, can be used to identify terminal posture application (such as horizontal/vertical screen switching, Dependent game, magnetometer pose calibrating), Vibration identification correlation function (such as pedometer, tap) etc.；It can also configure as terminal The other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, details are not described herein.

Voicefrequency circuit 606, loudspeaker, microphone can provide the audio interface between user and terminal.Voicefrequency circuit 606 can By the electric signal after the audio data received conversion, it is transferred to loudspeaker, voice signal output is converted to by loudspeaker；It is another The voice signal of collection is converted to electric signal by aspect, microphone, is converted to audio data after being received by voicefrequency circuit 606, then After the processing of audio data output processor 608, it is sent to such as another terminal through RF circuit 601, or by audio data Output is further processed to memory 602.Voicefrequency circuit 606 is also possible that earphone jack, with provide peripheral hardware earphone with The communication of terminal.

WiFi belongs to short range wireless transmission technology, and terminal can help user's transceiver electronics postal by WiFi module 607 Part, browsing webpage and access streaming video etc., it provides wireless broadband internet access for user.Although Figure 11 is shown WiFi module 607, but it is understood that, and it is not belonging to must be configured into for terminal, it can according to need do not changing completely Become in the range of the essence of invention and omits.

Processor 608 is the control centre of terminal, using the various pieces of various interfaces and the entire terminal of connection, is led to It crosses operation or executes the software program and/or module being stored in memory 602, and call and be stored in memory 602 Data execute the various functions and processing data of terminal, to carry out integral monitoring to terminal.Optionally, processor 608 can wrap Include one or more processing cores；Preferably, processor 608 can integrate application processor and modem processor, wherein answer With the main processing operation system of processor, user interface and application program etc., modem processor mainly handles wireless communication. It is understood that above-mentioned modem processor can not also be integrated into processor 608.

Terminal further includes the power supply 609 (such as battery) powered to all parts, it is preferred that power supply can pass through power supply pipe Reason system and processor 608 are logically contiguous, to realize management charging, electric discharge and power managed by power-supply management system Etc. functions.Power supply 609 can also include one or more direct current or AC power source, recharging system, power failure inspection The random components such as slowdown monitoring circuit, power adapter or inverter, power supply status indicator.

Although being not shown, terminal can also include camera, bluetooth module etc., and details are not described herein.Specifically in this implementation In example, the processor 608 in terminal can be corresponding by the process of one or more application program according to following instruction Executable file is loaded into memory 602, and the application program of storage in the memory 602 is run by processor 608, from And realize various functions：

Two dimensional image to be fused is obtained, and three-dimensional face model is constructed according to the face in two dimensional image to be fused；It obtains Template image, and obtain the rotation parameter of face in three dimensions in template image；Based on rotation parameter to three-dimensional people Face model is rotated, three-dimensional face model after being rotated；Human face three-dimensional model after rotation is converted into two-dimension human face image； Two-dimension human face image is merged with template image.

In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, it may refer to the detailed description above with respect to image interfusion method, details are not described herein again.

It will appreciated by the skilled person that all or part of the steps in the various methods of above-described embodiment can be with It is completed by instructing, or relevant hardware is controlled by instruction to complete, which can store computer-readable deposits in one In storage media, and is loaded and executed by processor.

For this purpose, the embodiment of the present invention provides a kind of storage medium, wherein being stored with a plurality of instruction, which can be processed Device is loaded, to execute the step in any image interfusion method provided by the embodiment of the present invention.For example, the instruction can To execute following steps：

The specific implementation of above each operation can be found in the embodiment of front, and details are not described herein.

Wherein, which may include：Read-only memory (ROM, Read Only Memory), random access memory Body (RAM, Random Access Memory), disk or CD etc..

By the instruction stored in the storage medium, any image provided by the embodiment of the present invention can be executed and melted Step in conjunction method, it is thereby achieved that achieved by any image interfusion method provided by the embodiment of the present invention Beneficial effect is detailed in the embodiment of front, and details are not described herein.

It is provided for the embodiments of the invention a kind of image interfusion method, device and storage medium above and has carried out detailed Jie It continues, used herein a specific example illustrates the principle and implementation of the invention, and the explanation of above embodiments is only It is to be used to help understand method and its core concept of the invention；Meanwhile for those skilled in the art, according to the present invention Thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be construed as Limitation of the present invention.

Claims

1. a kind of image interfusion method, which is characterized in that including：

The two-dimension human face image is merged with the template image.

2. image interfusion method according to claim 1, which is characterized in that described according in the two dimensional image to be fused Face construct three-dimensional face model the step of include：

Obtain default faceform；

According to the default faceform, three-dimensional coordinate of the face in the two dimensional image to be fused in three-dimensional planar is obtained Point, and three-dimensional face model is constructed according to the three-dimensional coordinate point.

3. image interfusion method according to claim 2, which is characterized in that it is described according to the default faceform, it obtains Three-dimensional coordinate point of the face in the two dimensional image to be fused in three-dimensional planar is taken, and is constructed according to the three-dimensional coordinate point The step of three-dimensional face model includes：

Obtain the first projective parameter that the face in the two dimensional image to be fused is projected in two-dimensional surface；

Obtain the second projective parameter that the default faceform is projected in two-dimensional surface；

According to first projective parameter and the second projective parameter, the face obtained in the two dimensional image to be fused is flat in three-dimensional Three-dimensional coordinate point in face；

Three-dimensional face model is constructed according to the three-dimensional coordinate point.

4. image interfusion method according to claim 1, which is characterized in that the face obtained in the template image The step of rotation parameter in three dimensions includes：

Threedimensional model is constructed according to the face in the template image；

Based on the rotation parameter of face in three dimensions in template image described in the obtaining three-dimensional model.

5. image interfusion method according to claim 4, which is characterized in that the face according in the template image Construct threedimensional model the step of include：

Obtain the third projective parameter that the face in the template image is projected in two-dimensional surface；

It obtains default faceform and the default faceform is projected in the 4th projective parameter in two-dimensional surface；

According to the third projective parameter and the 4th projective parameter, target of the face in template image in three-dimensional planar is obtained Three-dimensional coordinate point；

Threedimensional model is constructed according to the target three-dimensional coordinate point.

6. image interfusion method according to any one of claims 1 to 5, which is characterized in that described by the two-dimension human face The step of image is merged with the template image include：

Facial feature points detection is carried out to the two-dimension human face image, obtains the first human face characteristic point；

Facial feature points detection is carried out to the template image, obtains the second human face characteristic point；

According to first human face characteristic point and the second human face characteristic point, by the face fusion in the two-dimension human face image to institute State the face region in template image.

7. image interfusion method according to claim 6, which is characterized in that it is described according to first human face characteristic point and Second human face characteristic point, by face region of the face fusion in the two-dimension human face image into the template image Step includes：

According to first human face characteristic point, tessellation is carried out to the face in the two-dimension human face image, obtains first Face delta-shaped region；

According to second human face characteristic point, tessellation is carried out to the face in the template image, obtains the second face Delta-shaped region；

Affine transformation is carried out based on the first face delta-shaped region and the second face delta-shaped region, obtains target face three Angular domain；

The target face delta-shaped region is fused to the face region in the template image.

8. image interfusion method according to any one of claims 1 to 5, which is characterized in that when the X-Y scheme to be fused Face as in is positive face, and when the face in the template image is side face, the rotation parameter is in the template image The rotation parameter relative to positive face of side face in three dimensions, it is described to be based on the rotation parameter to the three-dimensional face model Rotated, after being rotated the step of three-dimensional face model include：

Positive face in the three-dimensional face model is rotated based on the rotation parameter, three-dimensional face mould after being rotated Type, the face after the rotation in three-dimensional face model are side face；

Described the step of being merged the two-dimension human face image with the template image includes：

Face on rear side of rotation in the two-dimension human face image is fused to the side face region in the template image.

9. a kind of image fusion device, which is characterized in that including：

Construction unit constructs three-dimensional for obtaining two dimensional image to be fused, and according to the face in the two dimensional image to be fused Faceform；

Acquiring unit, for obtaining the rotation of the face in template image, and the acquisition template image in three dimensions Parameter；

Rotary unit, for being rotated based on the rotation parameter to the three-dimensional face model, three-dimensional people after being rotated Face model；

10. image fusion device according to claim 9, which is characterized in that the construction unit includes：

First obtains subelement, for obtaining default faceform；

First building subelement, for according to the default faceform, the face obtained in the two dimensional image to be fused to exist Three-dimensional coordinate point in three-dimensional planar, and three-dimensional face model is constructed according to the three-dimensional coordinate point.

11. image fusion device according to claim 10, which is characterized in that the first building subelement is specifically used In：

12. image fusion device according to claim 9, which is characterized in that the acquiring unit includes：

Second building subelement, for constructing threedimensional model according to the face in the template image；

Second obtains subelement, for based on the rotation of face in three dimensions in template image described in the obtaining three-dimensional model Turn parameter.

13. according to the described in any item image fusion devices of claim 9 to 12, which is characterized in that the integrated unit includes：

First detection sub-unit obtains the first face characteristic for carrying out facial feature points detection to the two-dimension human face image Point；

Second detection sub-unit obtains the second human face characteristic point for carrying out facial feature points detection to the template image；

Subelement is merged, is used for according to first human face characteristic point and the second human face characteristic point, by the two-dimension human face image In face region of the face fusion into the template image.

14. image fusion device according to claim 13, which is characterized in that the fusion subelement is specifically used for：

15. a kind of storage medium, which is characterized in that the storage medium is stored with a plurality of instruction, and described instruction is suitable for processor It is loaded, the step in 1 to 8 described in any item image interfusion methods is required with perform claim.