CN103824269A - Face special-effect processing method and system - Google Patents

Face special-effect processing method and system Download PDF

Info

Publication number
CN103824269A
CN103824269A CN201210465767.2A CN201210465767A CN103824269A CN 103824269 A CN103824269 A CN 103824269A CN 201210465767 A CN201210465767 A CN 201210465767A CN 103824269 A CN103824269 A CN 103824269A
Authority
CN
China
Prior art keywords
face
picture
characteristic
effect processing
average
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201210465767.2A
Other languages
Chinese (zh)
Other versions
CN103824269B (en
Inventor
金连文
毛慧芸
朱武林
廖文鑫
张鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Samsung Guangzhou Mobile R&D Center
Samsung Electronics Co Ltd
Original Assignee
South China University of Technology SCUT
Samsung Guangzhou Mobile R&D Center
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT, Samsung Guangzhou Mobile R&D Center, Samsung Electronics Co Ltd filed Critical South China University of Technology SCUT
Priority to CN201210465767.2A priority Critical patent/CN103824269B/en
Publication of CN103824269A publication Critical patent/CN103824269A/en
Application granted granted Critical
Publication of CN103824269B publication Critical patent/CN103824269B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

Provided is a face special-efficiency processing method and system. The method includes: executing the following operations at a server end: receiving a picture from a client; detecting a face area from the received picture; detecting feature points from the detected face area; according to the shape of the detected face area and the positions of the feature points, searching for a group of similar face pictures from a photo database as a similar-picture subset; synthesizing an average face based on the similar-picture subset; and performing weight fusion on the synthesized average face and the detected face area according to a preset weight and using the result obtained through the fusion as a face picture with preset special-effect features; and sending the face picture, which is formed through the fusion and provided with the preset special-effect features, to the client.

Description

Face effect processing method and system
Technical field
The application relates to a kind of face effect processing method and system, relate in particular to a kind of face characteristic of the face image detect based on receiving from client computer, the similar pictures subset searching from the picture database of safeguarding is in advance to synthesize average face, to generate method and the system of special efficacy face.
Background technology
Face is to be subject to one of picture data type of extensive concern most.Modern age, psychologic research showed, if lineup's face shape is synthesized, the intermediate value shape (average face) obtaining tends to have larger attractive force.To 20 end of the centurys, have benefited from the flourish of computer technology and digital image processing techniques, emerge in large numbers again again about the psychological study of average face in a large number.Many psychologists have drawn the saying of " beautiful face is exactly average face ", they think, whether the shape of a face is tending towards on average, is the key that judges that whether this face is beautiful, " average face hypothesis " (the Averageness Hypothesis) that Here it is on psychology.
On the other hand, because the concept of anamorphose has proposed for many years, relative various algorithms have also obtained very large development, and are widely used in fields such as visual arts.External numerous about the software of anamorphose, as Photoshop, Sqirlz Morph, Fun Morph, Easy Morph etc., these softwares all allow user to specify characteristic of correspondence prototype on two given width pictures, then obtain anamorphose sequence by relevant anamorphose algorithm.Therefore, the user of rare needs can not adopt this class anamorphose software to ask for average face.But, adopt this type of software, conventionally each operation is merely able to two face pictures syntheticly, and these softwares can not provide a large amount of face materials for user's classification, these restrictions have affected user's experience all neither too much nor too littlely.
In addition, be particularly designed into face special effect transforming the website that average face technology is the theme little.The Face Research website (http://www.faceresearch.org) of Aberdeen of Britain made is more representational one.This website allows the photo that user uploads with oneself to calculate average face, and website is done perfectly not in man-machine interaction, and does not support the more property special effect processing effects to facial image.Meanwhile, because this website is developed by Englishman, the most face portraits in its database are American-European ethnic group, are not suitable for the use of Chinese user.
Summary of the invention
The object of the present invention is to provide a kind of system that the service of face special effect processing is provided, the special effect processing of average face and execution user face is provided from having the pictures of various special efficacys according to the unique point of the face image detect providing from client computer at server end, and the picture of special effect processing is sent to the client computer of request.
Another object of the present invention is to provide a kind of according to user's special efficacy requirement, the unique point of the face image detect based on providing from user is from having the synthetic average face of pictures of various special efficacys, and carry out method and the system of the special effect processing of user's face, thereby the service of the user's face special effect processing that meets user's requirement is provided.
According to an aspect of the present invention, provide a kind of effect processing method of face, comprise, operation below server end is carried out: A) receive picture from client computer; B) the image detect human face region from receiving; C) detect unique point from the human face region detecting; D) according to the shape of human face region and the position of unique point detected, search for one group of similar face picture as similar pictures subset from picture library; E) based on the synthetic average face of similar pictures subset; F) human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic; G) send to described client computer by merging the face picture with predetermined special efficacy characteristic forming.
According to a preferred embodiment of the invention, in steps A), also receiving additional information, described additional information comprises at least one in sex, age and race.
According to a preferred embodiment of the invention, at step D), according to shape, the position of unique point, the additional information of reception and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
According to a preferred embodiment of the invention, described effect processing method also comprises: H) the face picture that comprises characteristic point position is sent to described client computer, and receive through modifier face picture and additional information from client computer.Wherein, at step D) in, pass through modifier face picture as basis using what receive, carry out the search of similar face picture.
According to a preferred embodiment of the invention, described effect processing method also comprises: the information that receives at least one special efficacy characteristic from client computer.Wherein, for the each special efficacy characteristic receiving, perform step respectively D) to step F), to generate the face picture of the each described special efficacy characteristic with reception, and in step F) in, the whole face pictures with special efficacy characteristic that generate are sent to client computer.
According to a preferred embodiment of the invention, at step B) in, use AdaBoost people's face detection algorithm executor face to detect, and at step C) in, use active appearance models method to carry out feature point detection.
According to a preferred embodiment of the invention, step e) comprising: each unique point on human face characteristic point is carried out Delaunay triangulation and asked for convex closure, obtain multiple unique points; Analyze the aligning of executor's face shape based on broad sense Procrustes; Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape; It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
According to a preferred embodiment of the invention, carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
According to a preferred embodiment of the invention, carry out, in the average step of pixel-by-pixel weighting, carrying out described pixel-by-pixel weighting according to following formula average at each picture that conversion is obtained:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m,
Figure BDA00002414215700033
be the color value of the pixel that on k width picture, i is capable, j is listed as,
Figure BDA00002414215700034
for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
According to a preferred embodiment of the invention, in step F) in, carry out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure BDA00002414215700036
for i on average face image is capable, the color value of the pixel of j row,
Figure BDA00002414215700037
for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
According to a further aspect in the invention, provide a kind of effect processing method of face, comprising: A) from input image detect human face region; B) detect unique point from the human face region detecting; C) according to the shape of human face region and the position of unique point detected, search for one group of similar face picture as similar pictures subset from picture library; D) based on the synthetic average face of similar pictures subset; E) human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic.
According to a preferred embodiment of the invention, in steps A), also receiving the additional information of inputting, described additional information comprises at least one in sex, age and race.
According to a preferred embodiment of the invention, at step C), according to shape, the position of unique point, the additional information of input and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
According to a preferred embodiment of the invention, described effect processing method also comprises: F) the face picture that comprises characteristic point position is exported to user, and receive through modifier face picture and additional information from user. wherein, at step C) in, pass through modifier face picture as basis using what receive, carry out the search of similar face picture.
According to a preferred embodiment of the invention, described effect processing method also comprises: the information that receives at least one special efficacy characteristic, and for the each special efficacy characteristic receiving, perform step respectively C) to step e), to generate the face picture of the each described special efficacy characteristic with reception.
According to a preferred embodiment of the invention, in steps A) in, use AdaBoost people's face detection algorithm executor face to detect, and at step B) in, use active appearance models method to carry out feature point detection.
According to a preferred embodiment of the invention, step D) comprising: each unique point on human face characteristic point is carried out Delaunay triangulation and asked for convex closure, obtain multiple unique points; Analyze the aligning of executor's face shape based on broad sense Procrustes; Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape; It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
According to a preferred embodiment of the invention, carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
According to a preferred embodiment of the invention, carry out in the average step of pixel-by-pixel weighting at each picture that conversion is obtained, carry out described pixel-by-pixel weighting according to following formula and merge:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m,
Figure BDA00002414215700052
be the color value of the pixel that on k width picture, i is capable, j is listed as,
Figure BDA00002414215700053
for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
According to a preferred embodiment of the invention, in step e) in, carry out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure BDA00002414215700055
for i on average face image is capable, the color value of the pixel of j row, for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
According to a further aspect in the invention, provide a kind of special effect processing system of face, be included in server end and comprise with lower device: first device, for receiving picture from client computer; The second device, for the image detect human face region from receiving; The 3rd device, detects unique point for the human face region from detecting; The 4th device, for according to the shape of human face region and the position of unique point detected, searches for one group of similar face picture as similar pictures subset from picture library; The 5th device, for synthesizing average face based on similar pictures subset; The 6th device, for the human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic; The 7th device, for sending to described client computer by merging the face picture with predetermined special efficacy characteristic forming.
According to a preferred embodiment of the invention, first device also receives additional information, and described additional information comprises at least one in sex, age and race.
According to a preferred embodiment of the invention, the 4th device is according to shape, the position of unique point, the additional information of reception and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
According to a preferred embodiment of the invention, described special effect processing system also comprises: the 8th device, for the face picture that comprises characteristic point position is sent to described client computer, and receives through modifier face picture and additional information from client computer.Wherein, the 4th process modifier face picture that installs to receive, as basis, is carried out the search of similar face picture.
According to a preferred embodiment of the invention, first device also receives the information of at least one special efficacy characteristic from client computer.Wherein, for the each special efficacy characteristic receiving, the 4th device is according to the additional information of the position of the shape of the human face region detecting, unique point, reception and described special efficacy characteristic search similar pictures subset, the 5th device is based on the synthetic average face of similar pictures subset, and generate the face picture with described special efficacy characteristic, and the whole face pictures with special efficacy characteristic that generate are sent to client computer by the 6th device.
According to a preferred embodiment of the invention, the second device uses AdaBoost people's face detection algorithm executor face to detect, and the 3rd device uses active appearance models method to carry out feature point detection.
According to a preferred embodiment of the invention, the 5th device is when based on the synthetic average face of similar pictures subset: each unique point on human face characteristic point is carried out Delaunay triangulation and asked for convex closure, obtain multiple unique points; Analyze the aligning of executor's face shape based on broad sense Procrustes; Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape; It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
According to a preferred embodiment of the invention, carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
According to a preferred embodiment of the invention, to carry out described pixel-by-pixel weighting average according to following formula for the 5th device:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m,
Figure BDA00002414215700063
be the color value of the pixel that on k width picture, i is capable, j is listed as, for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
According to a preferred embodiment of the invention, the 6th device is carried out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure BDA00002414215700071
for i on average face image is capable, the color value of the pixel of j row, for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
According to a further aspect in the invention, provide a kind of special effect processing system of face, comprising: first device, for the image detect human face region from input; The second device, detects unique point for the human face region from detecting; The 3rd device, for according to the shape of human face region and the position of unique point detected, searches for one group of similar face picture as similar pictures subset from picture library; The 4th device, for synthesizing average face based on similar pictures subset; The 5th device, for the human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic.
According to a preferred embodiment of the invention, first device also receives the additional information of input, and described additional information comprises at least one in sex, age and race.
According to a preferred embodiment of the invention, the 3rd device is according to shape, the position of unique point, the additional information of input and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
According to a preferred embodiment of the invention, described special effect processing system also comprises: the 6th device, for the face picture that comprises characteristic point position is exported to user, and receives through modifier face picture and additional information from user.Wherein, the 3rd process modifier face picture that installs to receive, as basis, is carried out the search of similar face picture.
According to a preferred embodiment of the invention, described special effect processing system also comprises: the 7th device, for receiving the information of at least one special efficacy characteristic; And, for the each special efficacy characteristic receiving, the 3rd device is according to the additional information of the position of the shape of the human face region detecting, unique point, reception and described special efficacy characteristic search similar pictures subset, the 4th device is based on the synthetic average face of similar pictures subset, and the 5th device generates the face picture with described special efficacy characteristic.
According to a preferred embodiment of the invention, first device uses AdaBoost people's face detection algorithm executor face to detect, and the second device uses active appearance models method to carry out feature point detection.
According to a preferred embodiment of the invention, the 4th device is carried out Delaunay triangulation and asks for convex closure each unique point on human face characteristic point, obtains multiple unique points; Analyze the aligning of executor's face shape based on broad sense Procrustes; Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape; It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
According to a preferred embodiment of the invention, the 4th device is carried out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
According to a preferred embodiment of the invention, the 4th device is carried out described pixel-by-pixel weighting fusion according to following formula:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m, be the color value of the pixel that on k width picture, i is capable, j is listed as, for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
According to a preferred embodiment of the invention, the 5th device is carried out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure BDA00002414215700086
for i on average face image is capable, the color value of the pixel of j row, for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
Accompanying drawing explanation
By the description of carrying out below in conjunction with accompanying drawing, above and other object of the present invention and feature will become apparent, wherein:
Fig. 1 is the process flow diagram illustrating according to the face effect processing method of exemplary embodiment of the present invention;
Fig. 2 is the process flow diagram illustrating according to the face effect processing method of another exemplary embodiment of the present invention;
Fig. 3 is the process flow diagram illustrating according to the processing of the synthetic face average face of exemplary embodiment of the present invention;
Fig. 4 is the schematic diagram that human face characteristic point is shown;
Fig. 5 ~ Fig. 8 illustrates according to the special efficacy effect of the face effect processing method processing of exemplary embodiment of the present invention.
embodiment
Below, describe with reference to the accompanying drawings embodiments of the invention in detail.
Face effect processing method according to exemplary embodiment of the present invention can be embodied as to a kind of network service, at server end, according to the request from client computer, the picture receiving from client computer is carried out to face to be detected and facial feature points detection, the additional information providing according to the result of described detection computations and client computer, special efficacy effects etc. are searched for and the comparatively similar picture group sheet of face in the picture receiving from the picture library of server maintenance, synthesize average face with the similar picture searching, further to synthesize the picture with predetermined special efficacy feature, and synthetic special efficacy picture is sent to client computer.Like this, the terminal device (as mobile phone, panel computer etc.) that possesses finite computational abilities and storage capacity can make full use of the computational resource of server end and picture resource and obtain and meet the special effect processing picture that user requires.
According to another exemplary embodiment of the present invention, in the multi-purpose computer with certain computing power and storage capacity, realize face effect processing method of the present invention.
Can be implemented as software module, firmware or hardware module according to the each step in the effect processing method of the face of exemplary embodiment of the present invention, and described each step can be combined as to step still less, also arbitrary steps wherein can be split as to more step, or the operative combination in different steps can be become to new step, these combinations and fractionation all fall within the scope of the present invention.
Fig. 1 illustrates according to the process flow diagram of the face effect processing method of exemplary embodiment of the present invention.
With reference to Fig. 1, at step S110, server receives picture from client computer.Described server can directly receive described picture and additional information from client computer, also can receive described picture and additional information from client computer by another front-end server.According to an alternative embodiment of the invention, also receive and additional information, described additional information comprises at least one in sex, age and race.
At step S120, server detects human face region from the picture receiving.Specifically, for example, adopt the people's face detection algorithm based on class Haar feature and AdaBoost to realize face detection module, to realize, people face part and other irrelevant background information are made a distinction.
At step S130, server detects unique point from the human face region detecting.Can use any available feature point detecting method to carry out the detection of described human face characteristic point.According to exemplary embodiment of the present invention, the unique point of the many places such as the unique point that uses active appearance models (ASM) method to detect to comprise on face outline, eyes, eyebrow, nose, lip.In Fig. 4, mark the human face characteristic point detecting according to exemplary embodiment of the present invention.
After this, according to a preferred embodiment of the invention, at step S140, server has mark the face picture of each unique point of detection to send to described client computer, and at step S150, server receives through modifier face picture from described client computer.
At step S160, server, according to the shape of face and the position of unique point that detect, is searched for one group of similar face picture as similar pictures subset from picture library.According to exemplary embodiment of the present invention, one group of similar face picture that use k nearest neighbor algorithm has predetermined special efficacy characteristic (as beauty's type) from the database search of server maintenance is as similar pictures subset.According to a preferred embodiment of the invention, at step S160, server is according to shape, the position of unique point, the additional information of reception and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic can be in appearance characteristic, professionalism, identity properties, age characteristic.
At step S170, server is based on the synthetic average face of similar pictures subset.Fig. 3 is the process flow diagram illustrating according to the processing of the synthetic face average face of exemplary embodiment of the present invention, describes the processing of synthetic average face here with reference to Fig. 3 in detail.
With reference to Fig. 3, at step S1712, server is carried out Delaunay triangulation and asks for convex closure each unique point on human face characteristic point, obtains multiple unique points.According to exemplary embodiment of the present invention, for non-face region also can be processed in the affine stage of piecewise linearity, at 16 points of every width picture boundary uniform sampling, so in the Delaunay triangulation stage, obtain 59 points that need subdivision.Particularly, first set up a virtual large triangle as initial delta, guarantee that large triangle has comprised institute to be split a little, and be the initial triangle of cutting apart, then set up Delaunay triangulation by Incremental insertion, finally remove again all limits that are connected with initial delta, finally obtained the Delaunay triangulation net of these 59 points.
At step S1714, server is analyzed the aligning of executor's face shape based on broad sense Procrustes.Particularly, use characteristic point template carries out shape alignment to the each photo in given similar pictures subset, to eliminate size, position and the angle difference of people's face shape in different face pictures; Then, calculate average man's face shape, to carry out next step average face image calculation.Procrustes analyzes by by shape rotation, again by its size normalization, then by the barycenter displacement of two shapes to identical position, make the quadratic sum minimum of distance between the corresponding point of two shapes; In the time having multiple shape, need to aim at shape collection by the mode of iteration, until the each shape in shape collection is all less than significantly adjusting.
At step S1716, piecewise linearity affined transformation carried out by each face picture in similar pictures subset by server, so that the each picture after conversion has the characteristic point position consistent with average man's face shape.Can carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
At step S1718, it is average that each picture that server obtains conversion carries out pixel-by-pixel weighting, obtains average face picture.According to exemplary embodiment of the present invention, carry out described pixel-by-pixel weighting according to following formula average, thus synthetic average face picture:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m,
Figure BDA00002414215700113
be the color value of the pixel that on k width picture, i is capable, j is listed as,
Figure BDA00002414215700114
for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
After the synthetic average face of the processing by the step S1712 ~ S1718 shown in Fig. 3, at step S180, server is weighted fusion by the human face region of synthetic average face and detection according to predetermined weights, and the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic.According to exemplary embodiment of the present invention, server is carried out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure BDA00002414215700116
for i on average face image is capable, the color value of the pixel of j row,
Figure BDA00002414215700117
for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
When after the processing of completing steps S180, obtain having the face picture of predetermined special efficacy characteristic.The picture with various special efficacy effects that Fig. 5-Figure 11 illustrates respectively the original face picture of input/reception and generates according to face effect processing method of the present invention.Wherein, the right side of Fig. 5 illustrates that the original image in left side is through the effect picture of " beauty's type " special effect processing; The right side of Fig. 6 illustrates that the original image in left side is through the effect picture of " handsome boy's type " special effect processing; The right side of Fig. 7 illustrates that original Beckham's picture in left side is through the effect picture of " old man's face " special effect processing; The right side of Fig. 8 illustrates that the original image in left side is through the effect picture of " presidential face " special effect processing; The right side of Fig. 9 illustrates that the original image in left side is through the effect picture of " boss's face " special effect processing; The right side of Figure 10 illustrates that the original image in left side is through the effect picture of " children's face " special effect processing; The right side of Figure 11 illustrates that the original image in left side is through the effect picture of " baby's face " special effect processing.
At step S190, server sends to described client computer by merging the face picture with predetermined special efficacy characteristic forming.
According to an alternative embodiment of the invention, server receives the information of at least one special efficacy characteristic from client computer.Wherein, for the each special efficacy characteristic receiving, server performs step respectively S160 to step S180, to generate the face picture of the each described special efficacy characteristic with reception, and in step S190, the whole face pictures with special efficacy characteristic that generate are sent to client computer.
Fig. 2 is the process flow diagram illustrating according to the face effect processing method of another exemplary embodiment of the present invention.In the exemplary embodiment shown in Fig. 2, in the terminal device with certain computing power and memory capacity, realize described face effect processing method, described maintaining terminal equipment has and has all kinds of ages, the face picture database of occupation, identity, macroscopic features texts.
At the step S220 of Fig. 2, the image detect human face region that terminal device provides from user.Step S230, S260 ~ S280 respectively with Fig. 1 in step S130 and S160 ~ S180 corresponding.
At step S240, terminal device has mark the face picture of each unique point of detection to be shown to user by user interface, and at step S250, terminal device receives through modifier face picture by user interface.Described at Fig. 1, step S240 and S250 are not the steps that must carry out.
According to an alternative embodiment of the invention, terminal device also merges step S280 with the face Image Display with predetermined special efficacy characteristic forming to user.
Can be found out the description of exemplary embodiment of the present invention with reference to accompanying drawing by above-mentioned, can be according to user's special efficacy requirement according to face effect processing method of the present invention, there is predetermined special efficacy characteristic and the picture similar to user's face feature from the face pictures search of safeguarding in advance, and the similar picture based on searching and the photo that not only provides based on user oneself synthesizes average face, thereby face's picture of synthetic average face and user is merged to generate the face picture with predetermined special efficacy effect.The special efficacy face picture generating can be gathered the feature in user's self face feature and picture database with the similar pictures of certain effects.In addition, the present invention also can provide a kind of network service of face special effect processing.
It is a kind of for carrying out the system of face special effect processing that the present invention also provides.
Although represent with reference to preferred embodiment and described the present invention, it should be appreciated by those skilled in the art that in the case of not departing from the spirit and scope of the present invention that are defined by the claims, can carry out various modifications and conversion to these embodiment.

Claims (26)

1. an effect processing method for face, comprises, operation below server end is carried out:
A) receive picture from client computer;
B) the image detect human face region from receiving;
C) detect unique point from the human face region detecting;
D) according to the shape of human face region and the position of unique point detected, search for one group of similar face picture as similar pictures subset from picture library;
E) based on the synthetic average face of similar pictures subset;
F) human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic;
G) send to described client computer by merging the face picture with predetermined special efficacy characteristic forming.
2. effect processing method as claimed in claim 1, wherein, in steps A), also receive additional information, described additional information comprises at least one in sex, age and race.
3. effect processing method as claimed in claim 2, wherein, at step D), according to shape, the position of unique point, the additional information of reception and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
4. effect processing method as claimed in claim 1, also comprises:
H) the face picture that comprises characteristic point position is sent to described client computer, and receives through modifier face picture and additional information from client computer,
Wherein, at step D) in, pass through modifier face picture as basis using what receive, carry out the search of similar face picture.
5. effect processing method as claimed in claim 3, also comprises: receive the information of at least one special efficacy characteristic from client computer,
Wherein, for the each special efficacy characteristic receiving, perform step respectively D) to step F), to generate the face picture of the each described special efficacy characteristic with reception, and
In step F) in, the whole face pictures with special efficacy characteristic that generate are sent to client computer.
6. effect processing method as claimed in claim 1, wherein, at step B) in, use AdaBoost people's face detection algorithm executor face to detect, and at step C) in, use active appearance models method to carry out feature point detection.
7. the effect processing method as described in claim 3 or 4, wherein, step e) comprising:
Each unique point on human face characteristic point is carried out Delaunay triangulation and asked for convex closure, obtain multiple unique points;
Analyze the aligning of executor's face shape based on broad sense Procrustes;
Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape;
It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
8. effect processing method as claimed in claim 7, wherein, carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
9. effect processing method as claimed in claim 7, wherein, carries out, in the average step of pixel-by-pixel weighting, carrying out described pixel-by-pixel weighting average according to following formula at each picture that conversion is obtained:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m, be the color value of the pixel that on k width picture, i is capable, j is listed as,
Figure FDA00002414215600024
for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
10. effect processing method as claimed in claim 9, wherein, in step F) in, carry out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure FDA00002414215600026
for i on average face image is capable, the color value of the pixel of j row,
Figure FDA00002414215600027
for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
The effect processing method of 11. 1 kinds of faces, comprising:
A) the image detect human face region from inputting;
B) detect unique point from the human face region detecting;
C) according to the shape of human face region and the position of unique point detected, search for one group of similar face picture as similar pictures subset from picture library;
D) based on the synthetic average face of similar pictures subset;
E) human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic.
12. effect processing methods as claimed in claim 11, wherein, in steps A), also receive the additional information of inputting, described additional information comprises at least one in sex, age and race.
13. effect processing methods as claimed in claim 12, wherein, at step C), according to shape, the position of unique point, the additional information of input and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
The special effect processing system of 14. 1 kinds of faces, is included in server end and comprises with lower device:
First device, for receiving picture from client computer;
The second device, for the image detect human face region from receiving;
The 3rd device, detects unique point for the human face region from detecting;
The 4th device, for according to the shape of human face region and the position of unique point detected, searches for one group of similar face picture as similar pictures subset from picture library;
The 5th device, for synthesizing average face based on similar pictures subset;
The 6th device, for the human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic;
The 7th device, for sending to described client computer by merging the face picture with predetermined special efficacy characteristic forming.
15. special effect processing systems as claimed in claim 14, wherein, first device also receives additional information, and described additional information comprises at least one in sex, age and race.
16. special effect processing systems as claimed in claim 15, wherein, the 4th device is according to shape, the position of unique point, the additional information of reception and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
17. special effect processing systems as claimed in claim 14, also comprise:
The 8th device, for the face picture that comprises characteristic point position is sent to described client computer, and receives through modifier face picture and additional information from client computer,
Wherein, the 4th process modifier face picture that installs to receive, as basis, is carried out the search of similar face picture.
18. special effect processing systems as claimed in claim 16, first device also receives the information of at least one special efficacy characteristic from client computer,
Wherein, for the each special efficacy characteristic receiving, the 4th device is according to the additional information of the position of the shape of the human face region detecting, unique point, reception and described special efficacy characteristic search similar pictures subset, the 5th device is based on the synthetic average face of similar pictures subset, and generate the face picture with described special efficacy characteristic, and the whole face pictures with special efficacy characteristic that generate are sent to client computer by the 6th device.
19. special effect processing systems as claimed in claim 14, wherein, the second device uses AdaBoost people's face detection algorithm executor face to detect, and the 3rd device uses active appearance models method to carry out feature point detection.
20. special effect processing systems as described in claim 16 or 17, wherein, the 5th device when based on the synthetic average face of similar pictures subset,
Each unique point on human face characteristic point is carried out Delaunay triangulation and asked for convex closure, obtain multiple unique points;
Analyze the aligning of executor's face shape based on broad sense Procrustes;
Each face picture in similar pictures subset is carried out to piecewise linearity affined transformation, so that the each picture after conversion has the characteristic point position consistent with average man's face shape;
It is average that each picture that conversion is obtained carries out pixel-by-pixel weighting, obtains average face picture.
21. special effect processing systems as claimed in claim 20, wherein, carry out described piecewise linearity affined transformation according to following formula:
x ′ y ′ 1 = s x cos θ s x ( - sin θ ) t x s y sin θ s y cos θ t y 0 0 1 x y 1
Wherein, (x, y) is a bit in former coordinate system, (x ', y ') be the point in the coordinate system after conversion, θ is the anglec of rotation, s xfor the ratio of convergent-divergent in the horizontal direction, s ythe ratio of in the vertical direction convergent-divergent, t xand t ybe respectively in the horizontal direction with vertical direction on translational movement.
22. special effect processing systems as claimed in claim 20, wherein, it is average that the 5th device is carried out described pixel-by-pixel weighting according to following formula:
p i , j avg = 1 Σ k = 1 m w k Σ k = 1 m w k p i , j k
Wherein, m is the number that samples pictures is concentrated face picture, k=1 ... m,
Figure FDA00002414215600052
be the color value of the pixel that on k width picture, i is capable, j is listed as,
Figure FDA00002414215600053
for i on average face image is capable, the color value of the pixel of j row, w kthe weights of k width picture.
23. special effect processing systems as claimed in claim 22, wherein, the 6th device is carried out the Weighted Fusion of the human face region of synthetic average face and detection according to following formula:
y i , j = k 1 p i , j avg + k 2 f i , j avg
Wherein,
Figure FDA00002414215600055
for i on average face image is capable, the color value of the pixel of j row,
Figure FDA00002414215600056
for the i of human face region that detects is capable, the color value of the pixel of j row, k 1and k 2respectively the weights of giving the human face region of average face and detection.
The special effect processing system of 24. 1 kinds of faces, comprising:
First device, for the image detect human face region from input;
The second device, detects unique point for the human face region from detecting;
The 3rd device, for according to the shape of human face region and the position of unique point detected, searches for one group of similar face picture as similar pictures subset from picture library;
The 4th device, for synthesizing average face based on similar pictures subset;
The 5th device, for the human face region of synthetic average face and detection is weighted to fusion according to predetermined weights, the result that fusion is obtained is as the face picture with predetermined special efficacy characteristic.
25. special effect processing systems as claimed in claim 24, wherein, first device also receives the additional information of input, and described additional information comprises at least one in sex, age and race.
26. special effect processing systems as claimed in claim 25, wherein, the 3rd device is according to shape, the position of unique point, the additional information of input and the predetermined special efficacy characteristic of the human face region detecting, search for one group of similar face picture as similar pictures subset from picture library, described special efficacy characteristic comprises in appearance characteristic, professionalism, identity properties, age characteristic.
CN201210465767.2A 2012-11-16 2012-11-16 Face effect processing method and system Expired - Fee Related CN103824269B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210465767.2A CN103824269B (en) 2012-11-16 2012-11-16 Face effect processing method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210465767.2A CN103824269B (en) 2012-11-16 2012-11-16 Face effect processing method and system

Publications (2)

Publication Number Publication Date
CN103824269A true CN103824269A (en) 2014-05-28
CN103824269B CN103824269B (en) 2017-03-29

Family

ID=50759313

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210465767.2A Expired - Fee Related CN103824269B (en) 2012-11-16 2012-11-16 Face effect processing method and system

Country Status (1)

Country Link
CN (1) CN103824269B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106682632A (en) * 2016-12-30 2017-05-17 百度在线网络技术（北京）有限公司 Method and device for processing face images
CN106919885A (en) * 2015-12-24 2017-07-04 掌赢信息科技（上海）有限公司 A kind of face stylizing method and electronic equipment based on machine learning
CN107967463A (en) * 2017-12-12 2018-04-27 武汉科技大学 A kind of conjecture face recognition methods based on composograph and deep learning
CN108696699A (en) * 2018-04-10 2018-10-23 光锐恒宇（北京）科技有限公司 A kind of method and apparatus of video processing
CN108764048A (en) * 2018-04-28 2018-11-06 中国科学院自动化研究所 Face critical point detection method and device
CN108876705A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 Image synthetic method, device and computer storage medium
CN108876718A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 The method, apparatus and computer storage medium of image co-registration
CN109063560A (en) * 2018-06-28 2018-12-21 北京微播视界科技有限公司 Image processing method, device, computer readable storage medium and terminal
CN109952594A (en) * 2017-10-18 2019-06-28 腾讯科技（深圳）有限公司 Image processing method, device, terminal and storage medium
CN110189252A (en) * 2019-06-10 2019-08-30 北京字节跳动网络技术有限公司 The method and apparatus for generating average face image
CN110868598A (en) * 2019-10-17 2020-03-06 上海交通大学 Video content replacement method and system based on countermeasure generation network
CN111179174A (en) * 2019-12-27 2020-05-19 成都品果科技有限公司 Image stretching method and device based on face recognition points

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005069226A1 (en) * 2004-01-15 2005-07-28 British Telecommunications Public Limited Company Adaptive closed group caricaturing
CN101000688A (en) * 2007-01-15 2007-07-18 浙江大学 Method for automatic photomotage of multi-face

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005069226A1 (en) * 2004-01-15 2005-07-28 British Telecommunications Public Limited Company Adaptive closed group caricaturing
CN101000688A (en) * 2007-01-15 2007-07-18 浙江大学 Method for automatic photomotage of multi-face

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
郑南宁等: "人脸的表情与年龄变换和非完整信息的重构技术(上)", 《电子学报》 *

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106919885A (en) * 2015-12-24 2017-07-04 掌赢信息科技（上海）有限公司 A kind of face stylizing method and electronic equipment based on machine learning
CN106682632B (en) * 2016-12-30 2020-07-28 百度在线网络技术（北京）有限公司 Method and device for processing face image
CN106682632A (en) * 2016-12-30 2017-05-17 百度在线网络技术（北京）有限公司 Method and device for processing face images
CN109952594B (en) * 2017-10-18 2022-01-07 腾讯科技（深圳）有限公司 Image processing method, device, terminal and storage medium
US11043011B2 (en) 2017-10-18 2021-06-22 Tencent Technology (Shenzhen) Company Limited Image processing method, apparatus, terminal, and storage medium for fusing images of two objects
CN109952594A (en) * 2017-10-18 2019-06-28 腾讯科技（深圳）有限公司 Image processing method, device, terminal and storage medium
CN108876705A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 Image synthetic method, device and computer storage medium
CN108876718A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 The method, apparatus and computer storage medium of image co-registration
CN108876718B (en) * 2017-11-23 2022-03-22 北京旷视科技有限公司 Image fusion method and device and computer storage medium
CN108876705B (en) * 2017-11-23 2022-03-22 北京旷视科技有限公司 Image synthesis method and device and computer storage medium
CN107967463B (en) * 2017-12-12 2021-04-02 武汉科技大学 Virtual face recognition method based on synthetic image and deep learning
CN107967463A (en) * 2017-12-12 2018-04-27 武汉科技大学 A kind of conjecture face recognition methods based on composograph and deep learning
CN108696699A (en) * 2018-04-10 2018-10-23 光锐恒宇（北京）科技有限公司 A kind of method and apparatus of video processing
CN108764048B (en) * 2018-04-28 2021-03-16 中国科学院自动化研究所 Face key point detection method and device
CN108764048A (en) * 2018-04-28 2018-11-06 中国科学院自动化研究所 Face critical point detection method and device
CN109063560A (en) * 2018-06-28 2018-12-21 北京微播视界科技有限公司 Image processing method, device, computer readable storage medium and terminal
US11017580B2 (en) 2018-06-28 2021-05-25 Beijing Microlive Vision Technology Co., Ltd Face image processing based on key point detection
CN109063560B (en) * 2018-06-28 2022-04-05 北京微播视界科技有限公司 Image processing method, image processing device, computer-readable storage medium and terminal
CN110189252A (en) * 2019-06-10 2019-08-30 北京字节跳动网络技术有限公司 The method and apparatus for generating average face image
CN110189252B (en) * 2019-06-10 2023-08-18 北京字节跳动网络技术有限公司 Method and device for generating average face image
CN110868598A (en) * 2019-10-17 2020-03-06 上海交通大学 Video content replacement method and system based on countermeasure generation network
CN110868598B (en) * 2019-10-17 2021-06-22 上海交通大学 Video content replacement method and system based on countermeasure generation network
CN111179174A (en) * 2019-12-27 2020-05-19 成都品果科技有限公司 Image stretching method and device based on face recognition points
CN111179174B (en) * 2019-12-27 2023-11-03 成都品果科技有限公司 Image stretching method and device based on face recognition points

Also Published As

Publication number Publication date
CN103824269B (en) 2017-03-29

Similar Documents

Publication Publication Date Title
CN103824269A (en) Face special-effect processing method and system
Ohn-Bar et al. Joint angles similarities and HOG2 for action recognition
Gavrilova et al. Kinect sensor gesture and activity recognition: New applications for consumer cognitive systems
US9691184B2 (en) Methods and systems for generating and joining shared experience
Liao et al. Hand gesture recognition with generalized hough transform and DC-CNN using realsense
CN109118579A (en) The method, apparatus of dynamic generation human face three-dimensional model, electronic equipment
JP2006228061A (en) Face tracing program and face tracing method
CN111027403A (en) Gesture estimation method, device, equipment and computer readable storage medium
He Accelerometer Based Gesture Recognition Using Fusion Features and SVM.
WO2022120843A1 (en) Three-dimensional human body reconstruction method and apparatus, and computer device and storage medium
CN113254491A (en) Information recommendation method and device, computer equipment and storage medium
CN105096353A (en) Image processing method and device
Memiş et al. A kinect based sign language recognition system using spatio-temporal features
CN113361387A (en) Face image fusion method and device, storage medium and electronic equipment
Cao et al. What the back of the object looks like: 3D reconstruction from line drawings without hidden lines
CN110544287A (en) Picture matching processing method and electronic equipment
CN111833413B (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
Xu et al. A novel method for hand posture recognition based on depth information descriptor
Biswas et al. Smart city development: Theft handling of public vehicles using image analysis and cloud network
Wang Research on the evaluation of sports training effect based on artificial intelligence technology
CN112348069B (en) Data enhancement method, device, computer readable storage medium and terminal equipment
CN113392820A (en) Dynamic gesture recognition method and device, electronic equipment and readable storage medium
CN113902995A (en) Multi-mode human behavior recognition method and related equipment
Tubbs et al. Facial expression analysis as a means for additional biometric security in recognition systems
Colmenarez et al. Facial analysis from continuous video with applications to human-computer interface

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170329

Termination date: 20201116