CN107507217B

CN107507217B - Method and device for making certificate photo and storage medium

Info

Publication number: CN107507217B
Application number: CN201710706884.6A
Authority: CN
Inventors: 唐彦娜
Original assignee: Beijing Miji Technology Co ltd
Current assignee: Beijing Miji Technology Co ltd
Priority date: 2017-08-17
Filing date: 2017-08-17
Publication date: 2020-10-16
Anticipated expiration: 2037-08-17
Also published as: CN107507217A

Abstract

The invention provides a method, a device and a storage medium for manufacturing a certificate photo, wherein the manufacturing method comprises the steps of generating a standardized face image according to a user picture and determining a target model image; acquiring feature points and position information of the feature points of a first face area in a target model image and a second face area in a standardized face image; performing foreground segmentation processing on the first face area and the second face area; according to the position information of the feature points, firstly, carrying out global deformation processing on the standardized face image; then, carrying out local deformation processing on the first face area and the adjacent area thereof; and replacing the deformed first face region with the deformed second face region to obtain the replaced certificate photo. The method is accurate and reliable, can change the target model image for replacement to the minimum extent on the premise of ensuring that the characteristics of the standardized face image are not changed, improves the naturalization degree of the certificate photo, and ensures that the naked eye looks like no synthetic trace.

Description

Method and device for making certificate photo and storage medium

Technical Field

The invention relates to the technical field of image processing, in particular to a method and a device for manufacturing a certificate photo and a storage medium.

Background

In the current society, various certificates are just indispensable identity and right certificates in life and work of people; with the requirements of various certificates, the requirement of making certificates becomes an essential step in the certificate handling of people; in order to replace a tedious and time-consuming entity photographing mode, a software program for replacing a human face, which can be directly operated by a user, is also produced.

At present, two main methods for making identification photos based on face region replacement exist, wherein the first method is to extract a feature point set of a first face region image and a corresponding 'sectional region image'; calculating a feature point set in a second face region image and a corresponding 'mapping region image'; adjusting a 'mapping region image' of a second face region image according to the parameters of the feature point set of the first face region image to obtain a 'replacement mapping region image'; replacing the 'scratch area image' with the 'replacement map area image'; the second mode is to identify key parts of the region, including key parts of the first face region and key parts of the second face region; positioning key parts of the first face area and key parts of the second face area; calculating a motion vector field from the key part of the second face region to the key part of the first face region; deforming the second face area to the position of the first face area according to the motion vector field to obtain a deformed area; and performing natural treatment on the deformed region.

However, both of the above approaches have changed in geometry on the face region image that is used last; making the image shapes of the first face area and the second face area slightly different can not recognize which area the content used by the final synthesized image is provided by, and in some applications, keeping the information unchanged is a basic requirement; due to such technical drawbacks, this type of method is only used for applications of an amusement nature.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a method, a device and a storage medium for manufacturing a certificate photo, which have accurate and reliable manufacturing process, can change a target model image for replacement to the minimum extent on the premise of ensuring that the characteristics of a face image of a user are not changed, improve the naturalization degree of the certificate photo and enable the user to look like no synthetic trace.

In order to solve the technical problems, the invention provides the following technical scheme:

in a first aspect, the present invention provides a method of producing a document photo, the method comprising:

generating a standardized face image according to the received user picture, and determining a target model image;

respectively acquiring feature points and position information of the feature points of a first face region in the target model image and a second face region in the standardized face image;

foreground segmentation processing is respectively carried out on the first face area and the second face area in the target model image and the standardized face image;

according to the position information of the feature points of the first face area and the second face area, carrying out global deformation processing on the standardized face image, so that the difference value between the size and the position of the second face area in the deformed standardized face image and the size and the position of the first face area is in a preset range;

according to the position information of the feature points of the deformed first face area and the deformed second face area, carrying out local deformation processing on the first face area and the adjacent area thereof in the target model image, so that the geometric structures of the first face area and the second face area are the same; wherein the adjacent region is a region that is adjacent to the first face region, of an extended region centered on the first face region;

and replacing the deformed first face area with a deformed second face area to obtain a target model image comprising the second face area.

Further, before the generating a standardized face image from the received user picture, the method further comprises:

acquiring a material picture of the certificate photo;

sequentially checking each identification photo material picture, and setting all the identification photo material pictures passing the checking to be in the same picture format;

storing all the verified certificate photo material pictures with the same format in a certificate photo database;

correspondingly, the generating a standardized face image according to the received user picture and determining a target model image includes:

receiving a user picture;

verifying the user picture;

if the user picture passes the verification, generating a standardized face image according to the user picture;

receiving a model image selection instruction sent by a user, wherein the model image selection instruction comprises a corresponding identifier of a target model image in the certificate image database;

and calling a target model image from the certificate image database according to the corresponding identification in the model image selection instruction.

Further, the method further comprises:

displaying a target model image comprising the second face area;

if a secondary editing instruction is received, editing the target model image comprising the second face area according to the secondary editing instruction;

and if a developing instruction is received, sending the corresponding target model image comprising the second face area to a developing device according to the developing instruction, so that the developing device performs developing on the received target model image comprising the second face area.

Further, the obtaining of the feature points of the first face region in the target model image and the second face region in the standardized face image and the position information of the feature points respectively includes:

and respectively carrying out target area attribute detection on the target model image and the standardized face image, and respectively extracting each characteristic point of the first face area and the second face area and position information of each characteristic point.

Further, the performing foreground segmentation processing on the first face region and the second face region in the target model image and the standardized face image includes:

according to the structural relationship among the feature points of the first face area, segmenting the first face area and an environment background in the target model image to obtain the first face area and a mask of the first face area;

and segmenting a second face region and an environment background in the standardized face image according to the structural relationship among the feature points of the second face region to obtain the second face region and a mask of the second face region.

Further, the global deformation processing is performed on the standardized face image according to the position information of the feature points of the first face region and the second face region, so that the difference between the size and the position of the second face region in the deformed standardized face image and the size and the position of the first face region is within a preset range, and the global deformation processing includes:

according to the feature points of the first face area and the second face area, a geometric deformation model is established by applying a global deformation correction method;

carrying out global deformation processing on the standardized face image based on the geometric deformation model;

and correspondingly adjusting the position information of the characteristic points of the second face area and the mask of the second face area according to the deformation result of the standardized face image.

Further, the performing local deformation processing on the first face region and the adjacent region thereof in the target model image according to the position information of the feature points of the deformed first face region and the deformed second face region, so that the geometric structures of the first face region and the second face region are the same, includes:

respectively arranging a protection frame outside each feature point of a first face area in the target model image and a second face area in the standardized face image, wherein the position of the protection frame is constructed according to the center point of the protection frame, and the center point of the protection frame is the center of a closed convex hull area formed by peripheral outlines of all the feature points;

and according to the position information of the feature points of the deformed first face area and the deformed second face area, performing deformation processing on the first face area controlled by each feature point in a protection frame outside each feature point of the first face area in the target model image, so that the geometric structures of the first face area and the second face area are the same.

Further, replacing the deformed first face region with a deformed second face region to obtain a target model image including the second face region, including:

replacing the deformed first face region with a deformed second face region, wherein the replaced region is an intersection region of the mask of the deformed first face region and the mask of the deformed second face region, and a target model image comprising the second face region is obtained;

and performing natural fusion processing on the second face region in the target model image comprising the second face region.

In a second aspect, the present invention also provides a production apparatus for identification photographs, the production apparatus comprising: the device comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the steps of the method for manufacturing the certificate photo when executing the computer program.

In a third aspect, the invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the method of producing a certificate print.

According to the technical scheme, the invention provides the method, the device and the storage medium for manufacturing the identification photo, wherein the manufacturing method comprises the steps of generating a standardized face image according to a user picture and determining a target model image; acquiring feature points and position information of the feature points of a first face area in a target model image and a second face area in a standardized face image; performing foreground segmentation processing on the first face area and the second face area; according to the position information of the feature points, firstly, carrying out global deformation processing on the standardized face image; then, carrying out local deformation processing on the first face area and the adjacent area thereof; and replacing the deformed first face region with the deformed second face region to obtain the replaced certificate photo. The method is accurate and reliable, can change the target model image for replacement to the minimum extent on the premise of ensuring that the characteristics of the standardized face image are not changed, and improves the naturalization degree of the certificate photo. The invention ensures that the human face area of the certificate photo after replacement does not change, and solves the problem of unreal synthesized image caused by local area deformation and serious peripheral deformation of the local area in the technology for replacing the human face area in the certificate photo.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow chart of a method of making a security photograph of the present invention;

FIG. 2 is a schematic flow chart of steps A01-A03 before step 100 in the method for making a certificate photo of the present invention;

FIG. 3 is a schematic flow chart of step 100 of the method of making a certificate photo of the present invention;

FIG. 4 is a schematic diagram of the process of steps B01-B03 before and after step 600 in the method of making a certificate photo of the present invention;

FIG. 5 is a schematic flow chart of step 200 of the method of making a certificate photo of the present invention;

FIG. 6 is a schematic flow chart of step 400 of the method of making a certificate photo of the present invention;

FIG. 7 is a schematic flow chart of step 500 of the method of making a certificate photo of the present invention;

FIG. 8 is a schematic flow chart diagram of step 600 in a method of making a certificate photo of the present invention;

FIG. 9 is a schematic flow chart of a method for producing a certificate photo in an example of application of the present invention;

FIG. 10 is a schematic view showing the classification of the data of the identification photograph in the method of manufacturing the identification photograph in the application example of the present invention;

fig. 11 is a schematic diagram showing data types of face data input by a user in the method for manufacturing a certificate photo in the application example of the present invention;

FIG. 12 is a schematic view showing a flow of face replacement in the method of producing a certificate photograph in the application example of the present invention;

fig. 13 is a flowchart showing a method for verifying the feasibility of face replacement in the method for producing a certificate photograph in the application example of the present invention;

FIG. 14 is a schematic diagram of a system for producing a security photograph in accordance with the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Aiming at the defects of the certificate photo making case in the prior art, the invention provides the certificate photo making method, the certificate photo making method has the advantages that the face shape of a user is kept unchanged, the effect looks more natural and the user can be identified, and the method saves computing resources compared with a mode of converting from 2D to 3D and can ensure that the characteristics of a standardized face image are not changed.

An embodiment of the present invention provides a specific implementation manner of a method for manufacturing a certificate photo, and referring to fig. 1, the method for manufacturing a certificate photo specifically includes the following steps:

step 100: and generating a standardized face image according to the received user picture, and determining a target model image.

In step 100, the device for producing the identification photo generates a standardized face image from the received user picture and determines a target model image. It can be understood that after the target model image and the standardized face image are obtained respectively, the operation of replacing the face in the target model image with the face of the user can be performed.

Step 200: and respectively acquiring the characteristic points and the position information of the characteristic points of a first face area in the target model image and a second face area in the standardized face image.

In step 200, the device for making the identification photo respectively obtains feature points and position information of the feature points of a first face area in the target model image and a second face area in the standardized face image; it can be understood that the region attribute of the first face region in the target model image is the same as the region attribute of the second face region in the target model image, and the image obtained by the final replacement by the processor provided with the feature point acquisition unit is the background region of the target model image and the second face region in the standardized face image. The location information may be relevant location information including key features, contours, etc.

In order to accurately replace a first face region in a target model image with a second face region, after determining feature points and position information of the feature points of the first face region of the target model image and the second face region in a standardized face image, the first face region needs to be distinguished from an environment background of the target model image, and the first face region needs to be distinguished from the environment background of the standardized face image, so that the first face region and the second face region can be accurately processed and replaced.

Step 300: and performing foreground segmentation processing on the first face area and the second face area in the target model image and the standardized face image.

In step 300, the apparatus for making a certificate photo performs foreground segmentation processing on the first face region and the second face region in the target model image and the standardized face image. It can be understood that, the first face region and the second face region may be subjected to background segmentation processing according to a structural relationship between feature points; the characteristic points of the human face comprise positions of eyebrows, eyes, a nose, a mouth, a face contour and the like, so that the structural relationship among the characteristic points of the human face can be obtained according to the priori knowledge of the structure of the human face, and the priori knowledge of the structure of the human face is the time-space relationship of the interaction of facial muscles, namely the time-space relationship is determined by the anatomical structure of the human face and is not influenced by the imaging environment.

Since the contour of the first face region and the contour of the second face region are usually different, and the outside of the first face region of the reference image usually corresponds to a background applicable to different certificate requirements, after the first face region of the reference image and the second face region of the replacement image are obtained through the foreground segmentation processing in step 200, global adjustment on the contour of the second face region in the replacement image is required to be performed, so as to reduce the difference between the contours of the second face region and the first face region in the replacement image.

Step 400: and carrying out global deformation processing on the standardized face image according to the position information of the feature points of the first face area and the second face area, so that the difference between the size and the position of the second face area in the deformed standardized face image and the size and the position of the first face area is in a preset range.

In step 400, the device for manufacturing a certificate photo performs global deformation processing on the standardized face image according to the position information of the feature points of the first face region and the second face region, so that the difference between the size and the position of the second face region and the size and the position of the first face region in the deformed standardized face image is within a preset range. It can be understood that the preset range is determined in practical application according to practical situations, and the preset range is set according to a principle that the size of the second face area is as consistent as possible with the size of the first face area. Taking a human face as an example, in order to ensure the consistency of the size and the position, a Rigid Transformation (Rigid Transformation) can be used for calculating a Transformation matrix according to key coordinates of the human face in a target model image and a standardized human face image by using a least square method and a random sample invariant algorithm (RANSAC), and further mapping the standardized human face image into a target model image coordinate system.

Step 500: according to the position information of the feature points of the deformed first face area and the deformed second face area, carrying out local deformation processing on the first face area and the adjacent area thereof in the target model image, so that the geometric structures of the first face area and the second face area are the same; wherein the adjacent region is a region that is adjacent to the first face region, of an extended region centered on the first face region.

The extension area may be set to any shape according to an actual application, for example, the extension area is a rectangular area or a circular area.

In step 500, after performing global contour adjustment on the second face region in the standardized face image, in order to improve the accuracy of replacement between the first face region and the second face region, it is further necessary to perform local adjustment on the first face region in the target model image, that is, the identification photo making device performs local deformation processing on the first face region and its neighboring region in the target model image according to the position information of the deformed feature points of the first face region and the second face region, so that the geometric structures of the first face region and the second face region are the same. It can be understood that this step is an important step in the present application, and can simultaneously achieve the effects of fast manufacturing process and accurate replaced image. Wherein the geometric structure refers to the position of the outline and the characteristic points; and the same geometrical structure comprises: the first face area and the second face area have the same outline, and the positions of a certain characteristic point in the first face area and a corresponding characteristic point in the second face area are both located in a preset range aiming at the characteristic point; specifically, the position of a certain feature point eye of the face in the first face region and the position of a feature point eye corresponding to the face in the second face region are located in a preset range for the feature point eye; the preset range for the feature point eyes may be set to 3mm, that is, the difference between the position coordinates of a certain feature point eye of the face in the first face region and the position coordinates of a corresponding feature point eye of the face in the second face region is less than or equal to 3 mm.

Step 600: and replacing the deformed first face area with a deformed second face area to obtain a target model image comprising the second face area.

In step 600, after obtaining the globally deformed second face region and the locally deformed first face region, the apparatus for manufacturing a certificate photo replaces the deformed first face region with the deformed second face region, so as to obtain a target model image including the second face region. Here, it is assumed that the target model image is P, the normalized face image is Q, the region to be replaced is S, the non-replacement region is NS, and the final synthesized image is P' to satisfy the requirement

P’(S)＝Q(S)

P’(NS)＝P(NS)

From the above description, it can be seen that the method for manufacturing the identification photo in the embodiment of the present invention has a fast manufacturing process and high reliability, improves the accuracy of replacing the local region of the face image, can ensure that the manufactured face region is not changed, solves the problem of unreal synthesized image caused by local region deformation and severe deformation of the periphery of the local region in the technology for replacing the face region in the identification photo, and reduces the time consumption and cost of the manufacturing process.

The second embodiment of the present invention provides a specific implementation manner of step a01 to step a03 before step 100 in the method for manufacturing the identification photo, and referring to fig. 2, the steps a01 to a03 specifically include the following contents:

step A01: and acquiring the identification photo material picture.

Step A02: and sequentially checking each identification photo material picture, and setting all the identification photo material pictures passing the checking to be in the same picture format.

Step A03: and storing all the verified and same-format certificate photo material pictures in a certificate photo database.

Correspondingly, referring to fig. 3, the step 100 includes:

step 101: and receiving a user picture.

Step 102: verifying the user picture;

if the user picture passes the verification, entering step 103; otherwise, displaying verification failure information at the user terminal, and prompting the user about the shooting skill of the user picture.

In step 102, verifying the user picture specifically includes:

and judging whether the attribute information for inspection in the first face region and the attribute information for inspection in the corresponding second face region are in a preset attribute range or not according to the feature points and the position information of the feature points of the first face region and the second face region in the standardized face image, namely judging whether the standardized face image passes the inspection or not.

It can be understood that, according to the feature points of the first face region and the second face region in the standardized face image and the position information of the feature points, it is determined whether the attribute information to be checked of the first face region and the second face region is within an acceptable range, that is, it is determined whether the standardized face image passes the verification.

If yes, go to step 103.

Otherwise, step 106 is entered.

Step 103: generating a standardized face image according to the user picture;

step 104: and receiving a model image selection instruction sent by a user, wherein the model image selection instruction comprises a corresponding identifier of a target model image in the certificate image database.

Step 105: and calling a target model image from the certificate image database according to the corresponding identifier in the model image selection instruction.

Step 106: and judging that the second face area in the current standardized face image can not replace the first face area in the target model image, and outputting an informing prompt which can not be replaced. It can be understood that after the second face region in the current standardized face image is judged to be incapable of replacing the first face region in the target model image, an irreplaceable notification prompt is output, so that the user can obtain corresponding feedback information.

From the above description, it can be seen that the manufacturing process of the method for manufacturing the certificate photo in the embodiment of the present invention is complete and reliable, and the user experience of the user in the certificate photo manufacturing process is improved.

The third embodiment of the present invention provides a specific implementation manner of step B01 to step B03 after step 600 in the above method for manufacturing a certificate photo, and referring to fig. 4, the steps B01 to B03 specifically include the following contents.

Step B01: displaying a target model image comprising the second face area;

step B02: and if a secondary editing instruction is received, editing the target model image comprising the second face area according to the secondary editing instruction.

Step B03: and if a developing instruction is received, sending the corresponding target model image comprising the second face area to a developing device according to the developing instruction, so that the developing device performs developing on the received target model image comprising the second face area.

From the above description, it can be seen that the method for manufacturing a certificate photo in the embodiment of the present invention provides a reliable and fast method for verifying the replaceability of a local area, and if the verification is successful, the certificate photo is manufactured, so that the accuracy and the usability of the replacement result of the subsequent local area are ensured, the practicability and the reliability of the whole manufacturing method are improved, the integrity of the manufacturing process is improved, and the user experience of the user in manufacturing the certificate photo is improved.

An embodiment of the fourth embodiment of the present invention provides a specific implementation manner of step 200 in the above method for manufacturing a certificate photo, and referring to fig. 5, the step 200 specifically includes the following steps:

step 201: and respectively carrying out target area attribute detection on the target model image and the standardized face image, and determining the position information of the first face area and the second face area.

Step 202: and respectively extracting the feature points of the first face area and the second face area and the position information of the feature points.

In step 202, the operations of feature extraction, foreground segmentation, and the like of the model are usually completed when the database is manufactured, so that the response speed of the processing is faster, although the processing may be performed simultaneously, more computing resources are required for knowledge, the extraction processing is not required when the algorithm is locally processed, and the position information of the feature point is the position coordinate of the feature point in the area where the feature point is located. It is to be understood that feature point detection of the first face region and the second face region is first performed on both the input target model image and the normalized face image. The feature point detection is based on advanced human face detection to locate the position of the target model image and the standardized human face image.

For example, if the first face area and the second face area are both faces with front faces and no occlusion, a large number of face detection algorithms (AdaBoost, NPD, MTCNN, etc.) disclosed at present can achieve accurate face detection. After the face detection is finished, feature point extraction is carried out on the detected face, and methods for extracting the feature points include, but are not limited to, methods based on deep learning (such as a deep self-coding network, a deep regression network and the like) and active contour models (such as ASM, AAM and the like).

As can be seen from the above description, the method for manufacturing a certificate photo according to the embodiment of the present invention provides a method for reliably and quickly acquiring feature points and position information of the feature points of a first face region in a target model image and a second face region in a standardized face image, and the accuracy of the acquired feature points and position information of the feature points is high, so as to provide an accurate data base for the replacement of a subsequent local region.

In one embodiment, the method for making the identification photo includes one embodiment of step 300. The setting target in step 300 is to segment a first face region and an environment background in the target model image according to a structural relationship between feature points of the first face region to obtain a mask of the first face region and the first face region, and segment a second face region and an environment background in the standardized face image according to a structural relationship between feature points of the second face region to obtain a mask of the second face region and the second face region, where the mask specifically includes the following contents:

in general, the step 300 includes:

(1) generating foreground and background areas by inwards contracting and outwards expanding contour points of the first face area or the second face area; a first suspected foreground and background mask is obtained.

(2) And (4) segmenting the mask and the original image obtained in the previous step by using an oneCut algorithm, and further thinning the background and the foreground area to obtain a second suspected foreground and a background mask.

(3) And finally, processing the foreground region by using morphology (including opening operation, contour repair and filling of the sunken region) to obtain a final foreground and background mask.

Specifically, step 301: the suspected foreground and background regions are initialized. The auxiliary information is the position information of the key points of the face, and although the position information of the outline of the face is obtained in step 200, the accuracy of the position information is not enough, and the segmentation result cannot be directly generated, but the suspected foreground and background areas can be obtained as the initial segmentation areas by contracting and expanding the face outline inwards and outwards. Specifically, according to the structural relationship between the feature points of the first face region and the second face region, the first face region and the environmental background are initially segmented in the target model image, and the second face region and the environmental background are initially segmented in the standardized face image.

Step 302: on the basis of the primary segmentation, a plurality of efficient and accurate segmentation methods (such as graphCut, grabCut, oneCut, ACM and the like) can be utilized to perform secondary accurate segmentation. One preferred scheme is a one cut in grabcut algorithm, which is different from the traditional graph cutting algorithm, the algorithm does not need iteration, and the segmentation can be realized by one operation, so that a foreground region is obtained;

step 303: the foreground region optimized by post-processing comprises gray value limitation and morphological processing (opening and closing operation and hole filling), on one hand, the gray value similarity of the face region is guaranteed, and on the other hand, the integrity of the face contour is guaranteed.

In a specific embodiment, steps 400 to 500 of the present application can be specifically summarized as follows:

(1) carrying out overall position, direction and size transformation on the standardized face image;

(2) local geometric transformation is carried out on the reference image;

(3) in order to obtain a natural replacement result, the first face region and its neighboring regions are also adjusted accordingly.

The implementation process is specifically described in the following examples five and six:

an embodiment of the fifth embodiment of the present invention provides a specific implementation manner of step 400 in the above method for manufacturing a certificate photo, and referring to fig. 6, the step 400 specifically includes the following steps:

step 401: and establishing a geometric deformation model by applying an affine transformation mode in a global deformation correction method according to the feature points of the first face region and the second face region.

It is understood that the affine transformation is not the only transformation used in the present embodiment, and any transformation procedure in the global deformation correcting method may be implemented in the steps 401 to 403 in the present embodiment.

Step 402: and determining affine transformation parameters of the feature points of the second face region transformed to the feature points of the first face region according to a least square method based on the geometric deformation model.

Step 403: and deforming the standardized face image according to the affine transformation parameters, so that the difference value between the size and the position of a second face area and the size and the position of a first face area in the deformed standardized face image is in a preset range.

Step 404: and correspondingly adjusting the position information of the characteristic points of the second face area and the mask of the second face area according to the deformation result of the standardized face image.

In the above steps 401 to 404, the normalized face image is aligned with the target model image using the extracted feature points. For example, affine transformation parameters transformed from the feature points of the second face region to the feature points of the first face region are estimated by a least square method using the feature points of the first face region and the feature points of the second face region and affine transformation as a geometric deformation model. And the affine transformation parameters are utilized to deform the standardized face image to obtain the standardized face image which has the same size as the target model image and is basically matched with the positions of the first face area and the second face area. The same parameters are needed to deform the feature point coordinates of the mask and the second face region while deforming the standardized face image.

As can be seen from the above description, the method for manufacturing a certificate photo according to the embodiment of the present invention provides a method for reliably and quickly performing deformation processing on the standardized face image according to the feature points of the first face region and the second face region, so as to improve the practicability and reliability of the entire manufacturing method.

An embodiment of the sixth invention provides a specific implementation manner of step 500 in the above method for manufacturing the identification photo, and referring to fig. 7, the step 500 specifically applies a triangulation network construction method: (1) matching the feature points to enable key points of the first face area and the second face area to correspond one to one; (2) constructing a triangular network, and establishing a characteristic point mesh relation graph by utilizing algorithms such as Delaunay and the like; (3) refining the relation network obtained in the front by using a loop subdivision algorithm to construct a dense triangulation network; (4) performing affine parameter estimation and local interpolation on each sub-graph in the constructed mesh relation graph respectively to enable the feature points of the transformed first face region to be completely matched with the feature point position information of the second face region, wherein the method specifically comprises the following steps:

step 501: matching the feature points of the first face area and the second face area to ensure that the attributes expressed by the feature points with the same labels are the same; :

step 502: besides the key points and contour points in the first face area and the second face area, external control points are required to be added, and the target area and the neighborhood thereof are ensured to be in linkage change when the first face area is changed to the second face area.

Specifically, the specific setting manner of the extension area is as follows: firstly, calculating to obtain a defined positive rectangle which can just contain all the feature points; and then expanding the defined positive rectangle outwards according to the proportion of the defined positive rectangle to obtain the rectangular expanded region.

The feature points newly added for generating the expanded region are the multiple quantiles on the four vertexes and four sides of the expanded rectangle.

That is, the geometric structure of the face region and the peripheral background is ensured not to change by additionally adding some protective feature points, while the region outside the face region to the protective region is only slightly geometrically deformed, and the geometric deformation is weakened by the refined triangular mesh, so that the change is small and hardly noticeable to the naked eye.

Step 503: and constructing a triangular network by utilizing a Delaunay algorithm, and constructing a spatial position relationship among different feature points. Constructing a position relation network of the feature points of the first face area according to the position information of the N feature points of the first face area; constructing a position relation network of the feature points of the second face area according to the position information of the N feature points of the second face area;

step 504: in order to ensure the naturalness of the subsequent replacement effect, a more dense triangulation network is constructed on the basis of the original triangulation network by using a Loop subdivison algorithm. Specifically, the position relationship network of the first face region may be subdivided to obtain M feature points, so that the total number of the feature points of the first face region is M + N; subdividing the position relation network of the second face area to obtain M characteristic points, so that the total number of the characteristic points of the second face area is M + N; wherein N and M are both positive integers.

Step 505: after the construction of the triangulation network is completed, an affine transformation model can be constructed according to the positions of three corner points corresponding to each sub-network in the target model image and the standardized human face image, and the positions of the regions corresponding to the sub-networks of the target model image and the standardized human face image are completely consistent through projection.

In steps 501 to 505, more feature points are generated by using the relative coordinates of the feature points, the feature point coordinates after the geometric transformation of the second face region are used as a reference, and the face deformation of the model image is consistent with the face geometric structure of the second face region. The protection frame can be a circular frame or a rectangular frame, and the position of the protection frame is constructed according to the central point of the protection frame, wherein the central point of the protection frame is the center of a closed convex hull area formed by the peripheral outlines of all the characteristic points.

As can be seen from the above description, the method for manufacturing a certificate photo according to the embodiment of the present invention provides a method for reliably and quickly performing deformation processing on the first face region in the target model image according to the position information of the feature points of the deformed first face region and second face region, so as to improve the accuracy of replacing the local region, ensure that the local region after replacement is not changed, and solve the problem of unreal synthesized image caused by local region deformation and severe peripheral deformation of the local region in the local region replacement technology in the image.

An embodiment seventh of the present invention provides a specific implementation manner of step 600 in the above method for manufacturing a certificate photo, and referring to fig. 8, the step 600 specifically includes the following steps:

step 601: and replacing the deformed first face region with a deformed second face region, wherein the replaced region is an intersection region of the mask of the deformed first face region and the mask of the deformed second face region, and a target model image comprising the second face region is obtained.

Step 602: and performing natural fusion processing on the second face region in the target model image comprising the second face region.

Replacing a first face area of the deformed target model image by using the second face area, wherein the replaced second face area in the target model image comprising the second face area is the intersection of the two deformed first face areas and a second face area mask; and performing natural processing on a second face region in the target model image comprising the second face region to make the colors of the target model image comprising the second face region consistent. A feasible method is a Poisson editing algorithm which is a seamless splicing algorithm with excellent performance, and the method for solving the pixel optimal value by constructing a Poisson equation can well fuse the backgrounds of a source image and a target image while retaining the gradient information of the source image. Meanwhile, in order to accelerate the fusion speed, the input image is a minimum circumscribed rectangle containing a first face area and a second face area.

As can be seen from the above description, the method for manufacturing a certificate photo according to the embodiment of the present invention provides a method for reliably and quickly replacing a deformed first face region with a deformed second face region to obtain a target model image including the second face region, so as to improve the practicability and reliability of the entire manufacturing method.

It can be understood that the above-mentioned manufacturing apparatus may be an electronic device provided with a software program, an application program pre-installed in a device terminal, or a functional module pre-installed in an application program of a device terminal, where the device terminal includes but is not limited to a mobile phone, a tablet computer, a PC, an embedded system, or the like.

It can be understood that the order of implementation of the steps in the method for manufacturing the identification photo can be arbitrarily adjusted according to the actual application, and is not limited to the order of execution of the steps.

In order to further explain the scheme, the present invention further provides an application example of a method for manufacturing a certificate photo, and referring to fig. 9 to 13, in the application example, a target model image is specifically applied as a model image, a standardized face image is specifically applied as a user image, both the second face area and the second face area are front face images, and the application example specifically includes the following contents:

referring to fig. 9, the present application example is used to illustrate how to implement a marketing scheme of the identification photo service through an image face changing method, and people usually need formal clothing and beautiful hair styles when taking identification photos. However, through the invention, the user can generate beautiful certificate photo under the condition of only providing face image.

First, a large amount of identification photo material is prepared, whether the material can be face-changed is checked through a checking algorithm (checking is performed on a pair of images, it is assumed here that the user images are all positive images), and if the checking is passed, a specified format is generated and stored in a database, as shown in fig. 10.

When the user uploads an image, the image is verified (compared with a real face when the pose is compared), and if the verification is passed, a standardized face is generated, as shown in fig. 11.

The method comprises the steps that a formulated certificate photo is pushed to a user through an internet software platform (such as App), the system changes the face of a model after the user selects the picture of the model, then the picture is displayed to the user, if the user selects to dye hair or selects to change a background, corresponding processing is carried out on a background, and an image effect is displayed to the user. If the user likes the corresponding picture, the software platform can push the photo developing service to the user, and network marketing of the developing service is completed.

Referring to fig. 12, first, human face feature point detection and face segmentation are performed on both the input model image and the user image. The basis of the feature point detection is advanced human face detection to position the position of a human face image, and as the patent mainly aims at the front face without shielding, a large number of currently disclosed human face detection algorithms can realize accurate detection. After the face detection is finished, feature points are extracted from the detected face, the feature points generally refer to positions of eyebrows, eyes, a nose, a mouth and face contours of the face, and more methods for extracting the feature points comprise deep learning-based methods and active contour model-based methods.

And obtaining a preliminary segmentation of the human face foreground and the human face background by using the extracted human face characteristic points and the priori knowledge of the human face structure. On the basis of the preliminary segmentation, a plurality of efficient and accurate segmentation methods (such as graphCut, grabCut, oneCut, ACM and the like) can be used for segmenting the human face. The coordinates of the mask and the feature points of the face region can be obtained through the steps.

And then the extracted facial feature points are used for aligning the user image with the model image. For example: and estimating affine transformation parameters transformed from the user characteristic points to the model characteristic points by using the model characteristic points and the user characteristic points and by using affine transformation as a geometric deformation model through a least square method. And the affine transformation parameters are utilized to deform the user image to obtain the user image which has the same size with the model image and the face position is basically matched. The same parameters are needed to deform the feature point coordinates of the face mask and the user while deforming the user image.

Then, the feature point coordinates after the user geometric transformation are used as a reference, and the face deformation of the model image is consistent with the face geometric structure of the user. The problem then translates into a fine registration problem for images with known matching feature points. If the fine registration method is directly used at this time, for example, affine transformation using a local triangulation network or matching using a spline interpolation method, the image is unnatural, for example, the triangulation network method has a break of pixels between triangulation meshes, and the spline interpolation causes excessive distortion of the image. In order to eliminate the phenomenon, a protection rectangular frame is set outside the characteristic point area (by increasing the outline points of the rectangular frame into the characteristic points), and deformation is mainly carried out in the rectangular frame, so that the deformation range is effectively controlled. And then generating more characteristic points by using the relative coordinates of the characteristic points. For example, by constructing a triangulation network of feature points and subdividing the triangulation network by Loop subdivisions. The large number of control points after expansion can make the fine registration to obtain a smooth and natural image. The above deformation process is performed on the image of the model and the mask of the face of the model.

And finally, replacing the face of the model after deformation by using the face of the user, wherein the replaced area is the intersection of the face masks of the two images after deformation. And the replaced human face is subjected to naturalization processing, so that the colors of the images are consistent. The naturalization process may use a poisson fusion method or other image fusion methods.

Referring to fig. 13, the above methods are performed under the assumption that two images can be replaced naturally, however, any image containing two faces can not be replaced naturally. In order to improve the user experience, it is necessary to automatically detect whether two images can be replaced with each other. The condition for substitution is that the faces of the two images are substantially identical. The main factors causing the unnatural replacement of two images are the following aspects: a. whether or not to wear glasses; b. whether the postures of the human faces are consistent; c. whether the illumination is consistent. It is often difficult to deal with the situation when the user wears glasses naturally (taking into account the differences in the temples), especially at the ears, so that the user is not allowed to pass the verification if he wears glasses during the process.

Therefore, the method for detecting the glasses is needed in the checking method, because the position of the human face characteristic point can be obtained, the eye area image can be easily obtained, and then the model for judging whether the glasses are worn can be judged by training one model for judging whether the glasses are worn or not through the two-classification algorithm.

The pose problem is another main factor affecting the replacement effect, for example, when the model is a left-side face mask and the user is a front face, the whole image after replacement is very unnatural. Therefore, by calculating the parameters of the pose of the human face and only checking the image pair with smaller pose difference, 3 parameters representing the pose of the human head are respectively a yaw angle, a pitch angle and a roll angle, and the three parameters can be calculated and obtained through the coordinates of the feature points of the face.

The illumination factor is another factor mainly influencing face changing, the illumination distribution condition of the face can be alleviated through simple light intensity estimation and compensation when the distribution is uneven and the difference angle is small, and even discordance with the periphery of the face can be caused after a small amount of intensity difference compensation when the uneven degree is high.

From the above description, the application example of the present invention reduces the hardware cost of the system capable of implementing the production of the certificate photo, and the present invention has the advantages of flexible and rapid deployment, easy maintenance and capacity expansion, and provides more stable service. Better user experience, low calculation cost in the manufacturing process, high real-time performance and high image processing reality degree.

An eighth embodiment of the present invention provides a specific implementation manner of a system for manufacturing a certificate photo, which is capable of implementing all steps in the method for manufacturing a certificate photo, and referring to fig. 14, the system for manufacturing a certificate photo specifically includes the following contents:

and the standardized face image and target model image determining unit 10 is used for generating a standardized face image according to the received user picture and determining a target model image.

A feature point obtaining unit 20, configured to obtain feature points and position information of the feature points of a first face region in the target model image and a second face region in the standardized face image, respectively; wherein the region attribute of the first face region in the target model image is the same as the region attribute of the second face region in the target model image.

And a segmentation processing unit 30, configured to perform foreground segmentation processing on the first face region and the second face region in the target model image and the normalized face image.

And the standardized face image deformation unit 40 is configured to perform global deformation processing on the standardized face image according to the position information of the feature points of the first face region and the second face region, so that the difference between the size and the position of the second face region and the size and the position of the first face region in the deformed standardized face image is within a preset range.

A target model image deformation unit 50, configured to perform local deformation processing on a first face region and an adjacent region thereof in the target model image according to position information of feature points of the deformed first face region and second face region, so that the first face region and the second face region have the same geometric structure; wherein the adjacent region is a region that is adjacent to the first face region, of an extended region centered on the first face region.

And the local area replacing unit 60 is configured to replace the deformed first face area with a deformed second face area, so as to obtain a target model image including the second face area.

The embodiment of the system for manufacturing a certificate photo provided by the present invention can be specifically used for executing the processing flows of the above-mentioned embodiments of the manufacturing method, and the functions thereof are not described herein again, and reference can be made to the detailed description of the above-mentioned embodiments of the manufacturing method. From the above description, it can be seen that the process of manufacturing the certificate photo of the system for manufacturing the certificate photo in the embodiment of the present invention is fast and highly reliable, the accuracy of replacing the local region is improved, the local region after replacement is not changed, the problem of unreal composite image caused by local region deformation and severe local region peripheral deformation in the local region replacement technology in the image is solved, and the time consumption and cost of the manufacturing process are reduced.

An embodiment of the present invention provides a specific implementation manner of a device for manufacturing a certificate photo, which can implement all the steps in the method for manufacturing a certificate photo, and the device for manufacturing a certificate photo specifically includes the following contents:

a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the manufacturing methods when executing the computer program, such as implementing the steps of:

Step 200: and respectively acquiring the characteristic points and the position information of the characteristic points of a first human face area in the target model image and a second human face area in the standardized human face image.

Step 300: and performing foreground segmentation processing on the first face area and the second face area in the target model image and the standardized face image respectively.

From the above description, it can be seen that the manufacturing process of the manufacturing device of the identification photo in the embodiment of the present invention is fast and highly reliable, the accuracy of replacing the local region of the face image is improved, the manufactured face region can be ensured not to be changed, the problem of unreal synthesized image caused by local region deformation and serious deformation of the periphery of the local region in the technology of replacing the face region in the identification photo is solved, and meanwhile, the time consumption and the cost of the manufacturing process are reduced.

An embodiment of the present invention provides a specific implementation manner of a computer-readable storage medium capable of implementing all steps in the above method for manufacturing a certificate photo, where the computer-readable storage medium specifically includes the following contents:

the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods of manufacture, such as performing the steps of:

As can be seen from the above description, the manufacturing process of the computer-readable storage medium in the embodiment of the present invention is fast and highly reliable, the accuracy of replacing the local region of the face image is improved, it can be ensured that the manufactured face region is not changed, the problem of unreal synthesized image caused by local region deformation and severe deformation of the periphery of the local region in the face region replacement technology in the identification photo is solved, and the time consumption and cost of the manufacturing process are reduced.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The above examples are only for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method of producing a certificate photo, the method comprising:

according to the position information of the feature points of the first face area and the second face area, carrying out global deformation processing on the standardized face image, so that the difference value between the size and the position of the second face area in the standardized face image after global deformation and the size and the position of the first face area is in a preset range;

according to the position information of the feature points of the globally deformed second face region and the first face region, carrying out local deformation processing on the first face region and the adjacent region thereof in the target model image, so that the geometry structures of the locally deformed first face region and the globally deformed second face region are the same; wherein the adjacent region is a region that is adjacent to the first face region, of an extended region centered on the first face region;

replacing the deformed first face area with a deformed second face area to obtain a target model image comprising the deformed second face area;

according to the position information of the feature points of the second face region and the first face region after global deformation, the first face region and the adjacent region in the target model image are subjected to local deformation processing, so that the geometric structure of the first face region after local deformation is the same as that of the second face region after global deformation, and the method comprises the following steps:

and according to the position information of the feature points of the second face area and the first face area after global deformation, carrying out deformation processing on the first face area controlled by each feature point in a protection frame outside each feature point of the first face area in the target model image, so that the geometric structures of the first face area after local deformation and the second face area after global deformation are the same.

2. The method of claim 1, wherein prior to said generating a standardized image of a face from a received picture of a user, the method further comprises:

acquiring a material picture of the certificate photo;

receiving a user picture;

verifying the user picture;

3. The method of manufacturing according to claim 1, further comprising:

displaying a target model image comprising the second face area;

4. The method according to claim 1, wherein the obtaining of the feature points and the position information of the feature points of the first face region in the target model image and the second face region in the standardized face image respectively comprises:

5. The production method according to claim 1, wherein performing foreground segmentation processing on the first face region and the second face region in the target model image and the normalized face image comprises:

6. The method according to claim 1, wherein the global deformation processing is performed on the normalized face image according to the position information of the feature points of the first face region and the second face region, so that the difference between the size and the position of the second face region and the size and the position of the first face region in the deformed normalized face image is within a preset range, and the method includes:

7. The method according to claim 1, wherein the replacing the deformed first face region with a deformed second face region to obtain a target model image including the deformed second face region comprises:

replacing the deformed first face region with a deformed second face region, wherein the replaced region is an intersection region of the mask of the deformed first face region and the mask of the deformed second face region, and a target model image comprising the deformed second face region is obtained;

and performing natural fusion processing on the deformed second face region in the target model image comprising the deformed second face region.

8. A device for producing a certificate photo, the device comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the production method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.