KR102240237B1 - Apparatus and method for facial reproduction using genetic information - Google Patents

Abstract

The present invention discloses an apparatus and method for reproducing a face using genetic information. A face reproducing apparatus using genetic information according to an aspect of the present invention can more effectively estimate a face of a missing person by estimating a person's face based on the genetic information extracted from the biometric information, thereby more effectively estimating the disappearance event. Can be solved.

Description

Facial reproduction device and method using genetic information {APPARATUS AND METHOD FOR FACIAL REPRODUCTION USING GENETIC INFORMATION}

The present invention relates to a face reproducing apparatus and method using genetic information, and more particularly, a face reproducing apparatus using genetic information for reproducing a human face based on the extracted genetic information by extracting genetic information from human biometric information. And a method.

In the prior art, there has been a case of estimating a face of a long-term missing person and a criminal by using a 3D face model or the like to reproduce a face, or estimating the face of a missing person and a criminal by comparing previously stored DNA information and acquired DNA information. However, in order to reproduce the face according to the prior art, it was possible only when DNA information for contrast was stored in advance, and the face reproduced according to the prior art using a 3D face model follows the statement of an eyewitness. Due to the large difference, there is a problem that the practical use is degraded.

Korean Patent Publication No. 2004-0009460 (published on Jan. 31, 2004)

The present invention has been proposed to solve the above problems, extracting genetic information from human biometric information such as blood, sweat, skin cells, hair, etc., and reproducing a human face based on the extracted genetic information. An object of the present invention is to provide an apparatus and method for reproducing a face based on biometric information.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by an embodiment of the present invention. In addition, it will be easily understood that the objects and advantages of the present invention can be realized by the means shown in the claims and combinations thereof.

In order to achieve the above object, a face reproduction apparatus based on biometric information according to an aspect of the present invention includes: a gene information analysis unit for analyzing gene information from an acquired biological sample; SNP extraction unit for extracting SNP information from the analyzed gene information; A deep learning learning unit that learns the genetic information through a machine learning technique; And a face reproducing unit for reproducing a face image based on the extracted SNP information.

It characterized in that it further comprises a; correction unit for correcting the face image estimated based on the information stated by the witness or the CCTV image.

The genetic information analysis unit may extract at least one of sex, race, and lineage information from the analyzed genetic information.

The deep learning learning unit is characterized in that it learns genetic information using a convolutional neural network (CNN) model.

A method for reproducing a face in a face reproducing apparatus using genetic information according to another aspect of the present invention for achieving the above object includes: analyzing the genetic information in the obtained biological sample; Extracting SNP information from the analyzed gene information; Learning the genetic information through a machine learning technique; And reproducing a face image based on the extracted SNP information.

It characterized in that it further comprises a; step of correcting the estimated face image based on the information stated by the witness or the CCTV image.

The step of analyzing the genetic information is characterized in that at least one of information on sex, race, and pedigree is extracted from the analyzed genetic information.

The step of learning the genetic information through a machine learning technique is characterized in that the genetic information is learned using a convolutional neural network (CNN) model.

According to an aspect of the present invention, by estimating a person's face based on the genetic information extracted from the biometric information, the face of the missing person can be more effectively estimated, and thus the disappearance case can be more effectively resolved.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention pertains from the following description. .

The following drawings appended to the present specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention along with specific details for carrying out the present invention. It should not be interpreted as being limited to the stated matters.
1 is a schematic functional block diagram of a face reproducing apparatus according to an embodiment of the present invention;
2 is a schematic functional block diagram of a face reproducing apparatus according to another embodiment of the present invention;
3 is a schematic flowchart of a method for reproducing a face according to an embodiment of the present invention.

The above-described objects, features, and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, whereby those of ordinary skill in the technical field to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that a detailed description of known technologies related to the present invention may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, “… A term such as “sub” means a unit that processes at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software.

The face reproducing apparatus according to the present invention can generate a montage based on the biometric information left at the scene when it is difficult to cover the suspect because a clue about the face of a witness or a criminal through CCTV is not given at a crime scene.

1 is a schematic functional block diagram of a face reproducing apparatus according to an embodiment of the present invention, and FIG. 2 is a schematic functional block diagram of a face reproducing apparatus according to another embodiment of the present invention.

Referring to FIG. 1, the face reproduction apparatus 100 according to the present embodiment includes a genetic information analysis unit 110, an SNP extraction unit 120, a deep learning learning unit 130, and a face reproduction unit 140.

The gene information analysis unit 110 analyzes gene information from the obtained biological sample. In this case, the biological sample may include human biometric information such as blood, sweat, skin cells, hair, and the like. At this time, at least one of information such as sex, race, and pedigree may be extracted from the analyzed genetic information. Meanwhile, the gene information analysis unit 110 may store the analyzed gene information in a database, which is a separate storage space, and convert it into big data. For example, the gene information analysis unit 110 may collect gene information and face images of people corresponding thereto, and use the collected information to reproduce the face image. Meanwhile, in the database, facial image information and DNA information are collected in advance, feature components related to the face shape are extracted from the facial image information, SNP information that affects the appearance of the face is extracted from the DNA information, and the extracted face shape is A correlation diagram modeled through statistical analysis between attribute values for each site and SNP information may be stored.

The SNP extraction unit 120 may extract SNP information from the analyzed gene information. Single Nucleotide Polymorphism (SNP) may be information that affects the appearance of a face. On the other hand, genes are inherited according to the order in which the four bases of DNA, namely adenine (A), guanine (G), cytosine (C), and thymine (T), are arranged in the DNA by the nucleotide sequence method on the DNA. Save the information. After all, it can be said that a gene is an explanatory text composed of four alphabets (A, G, C, T). At this time, a genetic change or mutation showing a difference in one nucleotide sequence (A, T, G, C) in the DNA sequence is called SNP. The location where two or more alleles present at a frequency of 1% or more in the population occurs is called SNP, and when the allele is present at a frequency of 5% or more, it is called a common polymorphism, and when 1 to 5% is a rare polymorphism. Classify. SNP is a part that shows a lot of variation for each individual, so it is often used for DNA fingerprint analysis. In this embodiment, it is used to estimate facial reproduction based on the SNP that affects the appearance of the face.

The deep learning learning unit 130 learns genetic information converted to big data through machine learning techniques. The deep learning learning unit 130 may learn stored data in a predetermined ratio. In this case, a certain ratio may be 8:2, and training may be performed by using 8 as the training data and 2 as the test data. The machine learning technique is a known technique and a detailed description thereof will be omitted. During training, a CNN (Convolutional Neural Network) can be used as a deep learning model, and when a CNN (Convolutional Neural Network) is used, genetic information can be used as an input, and face data can be used as an output. In more detail, the deep learning learning unit 130 may learn and model the relationship between the genetic information and the face image through a machine learning technique and store it in a database, which is a separate storage space. In this case, the modeled information may be expressed as a relational expression between the face image and SNP information. Meanwhile, the database may further store a face image and image information for each portion of the face. The database can store data suitable for various races and countries.

The face reproducing unit 140 may estimate a human face image based on the SNP information. That is, the face reproducing unit 140 may estimate the human face image by matching the SNP information extracted by the SNP extraction unit 120 with the modeled information learned by the deep learning learning unit 130. In other words, the face reproducing unit 140 may estimate the face image by reflecting the extracted SNP information to the modeled information by learning through a machine learning technique.

Meanwhile, the face reproducing apparatus 100 according to the present embodiment may further include a correction unit 210.

The correction unit 210 may correct the face image estimated through the face reproducing unit 140 based on the information stated by the witness and/or the image information of the CCTV. Accordingly, it is possible to correct the face image by adding indirect evidence such as a witness or CCTV to the face of the suspect estimated using biometric information, which is direct evidence. In this case, there is an effect of increasing the accuracy of a person's face image.

In addition, the face reproducing apparatus 100 according to the present embodiment may further include a modeling unit (not shown).

The modeling unit may model a correlation through statistical analysis between the attribute value of each part of the face shape obtained by classifying the face shape and SNP information, and the modeling may be expressed as a relational expression between the face shape and SNP information. Thereafter, when the correlation between the face shape and the SNP information is modeled in the modeling unit, the face reproducing unit 130 may estimate the face using the genetic information from the biological sample input thereafter. When the face reproducing unit 130 estimates the shape of the face using the face shape generated by the modeling unit, the SNP information extracted by the SNP extraction unit 120 is used as the face shape and SNP information generated by the modeling unit (not shown). It is possible to estimate the shape of the face by inputting it into the analysis model of the correlation between the face and predicting what attribute value the facial region has for each classified attribute. In this case, the feature component related to the shape of the face may be extracted by a separate facial feature extraction unit (not shown). As for the criterion of facial features, attribute values may be defined by the user, and each attribute value may be displayed in stages and used. Meanwhile, the attribute value of the expressed face may be variably changed.

Meanwhile, the face reproducing unit 140 estimates the human face image based on the SNP information, but the SNP information extracted by the SNP extraction unit 120 is correlated between the face shape generated by the modeling unit (not shown) and the SNP information. It is also possible to estimate the shape of the face by inputting it into the analysis model of, and predicting what attribute value the face part has for each attribute classified by the facial feature extraction unit (not shown).

In addition, the face reproducing apparatus according to the present embodiment further includes a face estimating unit (not shown) for estimating an initial face shape before estimating the face shape, and applying a weight to an attribute value for each face part to the estimated initial face. Thus, by merging the shape of each part of the face, the face can be reproduced more accurately.

The face reproducing apparatus according to the above can reproduce a human face in the form of a two-dimensional face model. In this case, according to another embodiment of the present invention, the face reproducing apparatus may further include a three-dimensional model forming unit (not shown) that reproduces the reproduced two-dimensional face model shape in a three-dimensional shape to reproduce a three-dimensional face shape. .

The 3D model forming unit may generate a 3D face model through a linear combination of weights between 3D standard face types by extracting feature points from the 2D face model generated by the face reproducing unit 140. Specifically, feature points for eyes, nose, mouth, and facial contours are extracted from the input 2D face image, and based on the extracted 2D feature points, the target is based on a linear combination of weights between 3D standard face types representing Koreans. You can create a three-dimensional face of an image. In this case, the generated 3D face may be generated based on various Korean standard models.

In addition, the 3D model forming unit may provide at least one function of automatic facial expression change, free transformation provision, background change provision, and decoration provision. The automatic expression change function is a function that sets various facial expressions of a 3D face model through value adjustment within a category, and provides an easy way to handle functions manually performed by an expert. This is a technique that automatically generates an expression model by transferring the expression information of the previously constructed 3D standard models to the generated 3D montage model.It constructs a 3D basic set of various facial expressions and uses a linear combination of an expressionless model and an appropriate weight. You can create the expression you want. By applying the created facial expression source to the target montage 3D model, it is possible to create a target model with a changed facial expression, which is the final purpose.

A feature point-based method or a sketch-based method can be used for the function of providing free transformation. The feature point-based free transformation is a 3D model editor that sets the feature points for each part in the 3D model and transforms the face model for each part through free transformation (rotation, movement, scale) of the set points. The feature point is set at the position that best expresses the face information, and the maximum deformation is provided only with the set feature point, but it is possible to provide the user with easy-to-use deformation functions.

Sketch-based free transformation is an editor technology that transforms the shape of the silhouette of the face by drawing the silhouette of the face that the user wants to edit directly and the modified silhouette when transforming the 3D model. In particular, through a sketch-based transforming user interface, it is possible to delicately transform a face shape expressed as a silhouette, and at the same time, it is possible to provide the user with maximum freedom in transforming a face model.

The background change provision function gives realism to the image visible in the 3D model through transformation of the space in which the 3D model is visible, and enables a more diverse situation to be reproduced, thereby inducing the creation of a more similar to the target face. . Accordingly, background conversion items include background light source adjustment and color adjustment, which have the greatest influence on the image of the face. In addition, it is also possible to provide various rendering techniques to adjust the feeling of viewing the overall image.

The decoration provision function can be used to decorate various styles by wearing various accessories on the 3D model. This becomes a necessary factor to create a shape more similar to the target face in consideration of the fact that the image of the face changes a lot according to the style. Three-dimensional decorating items include glasses, hats, hair, and accessories. Each prop can be properly positioned on the target model through position movement and scale transformation by the editor tool.

3 is a schematic flowchart of a method for reproducing a face according to an embodiment of the present invention.

Referring to FIG. 3, first, the face reproducing apparatus 100 analyzes gene information from an acquired biological sample (S310). In this case, the biological sample may include human biometric information such as blood, sweat, skin cells, hair, and the like. In this case, the face reproducing apparatus 100 may extract at least one of information such as sex, race, and blood line from the analyzed genetic information. Meanwhile, the face reproducing apparatus 100 may store the analyzed gene information in a database, which is a separate storage space, and convert it into big data.

Next, the face reproducing apparatus 100 extracts SNP information from the analyzed gene information (S320). Single Nucleotide Polymorphism (SNP) may be information that affects the appearance of a face.

Next, the face reproducing apparatus 100 learns genetic information through a machine learning technique (S330). During training, a CNN (Convolutional Neural Network) can be used as a deep learning model, and when using a CNN (Convolutional Neural Network), genetic information can be used as an input, and face data can be used as an output.

Next, the face reproducing apparatus 100 reproduces the face image based on the extracted SNP information (S340). The face reproducing apparatus 100 may estimate the human face image by matching the extracted SNP information with the modeled information by learning by a machine learning model.

Meanwhile, the face reproducing apparatus 100 corrects the face image estimated based on the information stated by the witness or the CCTV image (S350).

The present invention is to reproduce a face based on the genetic information included in the biometric information, and the biometric information of people, that is, the genetic information that can be extracted through blood, sweat, skin cells, hair, etc., is converted into big data, and then converted into big data. After learning the generated genetic information through machine learning techniques, the human face can be reproduced using the biometric information of a person whose face is unknown. Accordingly, it can be used to estimate the face of a missing person and/or a criminal, or to reproduce an ancient person.

The methods according to the embodiments of the present invention may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components, and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and constructed for the present invention, and may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. media) and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa.

While this specification includes many features, such features should not be construed as limiting the scope or claims of the invention. In addition, features described in separate embodiments of the present specification may be combined and implemented in a single embodiment. Conversely, various features described in a single embodiment of the present specification may be individually implemented in various embodiments, or may be properly combined and implemented.

Although the operations have been described in a specific order in the drawings, it should not be understood that such operations are performed in a specific order as shown, or as a series of consecutive orders, or that all described operations are performed to obtain a desired result. Multitasking and parallel processing can be advantageous in certain environments. In addition, it should be understood that classification of various system components in the above-described embodiments does not require such classification in all embodiments. The above-described app components and systems may generally be implemented as a package in a single software product or multiple software products.

The present invention described above, for those of ordinary skill in the art to which the present invention pertains, various substitutions, modifications and changes are possible within the scope of the technical spirit of the present invention. It is not limited by the drawings.

100: face reproduction device
110: genetic information analysis unit
120: SNP extraction unit
130: Deep Learning Department
140: face reproduction unit
210: correction unit

Claims (8)

A gene information analysis unit that analyzes gene information from the obtained biological sample;
SNP extraction unit for extracting SNP information from the analyzed gene information;
A deep learning learning unit that learns the genetic information through a machine learning technique; And
Including; a face reproducing unit for reproducing a face image based on the extracted SNP information,
A modeling unit for modeling a correlation through statistical analysis between an attribute value for each part of the face shape obtained by classifying the face shape and SNP information; And
A face estimating unit for estimating the initial face shape before estimating the face shape; further comprising,
The face reproduction unit,
After the correlation between the face shape and the SNP information is modeled in the modeling unit, the face is estimated using genetic information from the input biological sample,
The face estimation unit,
A face reproduction apparatus using genetic information, characterized in that the face is reproduced by merging the shape of each face by applying a weight to an attribute value of each face part to the estimated initial face. The method of claim 1,
A face reproduction apparatus using genetic information, further comprising: a correction unit for correcting a face image estimated based on information stated by a witness or a CCTV image. The method of claim 1,
The genetic information analysis unit,
Facial reproduction apparatus using genetic information, characterized in that extracting at least one of information on sex, race, and pedigree from the analyzed genetic information. The method of claim 1,
The deep learning learning unit,
A facial reproduction device using genetic information, characterized in that it learns genetic information using a CNN (Convolutional Neural Network) model. In the face reproduction method in the face reproduction apparatus using genetic information,
Analyzing gene information from the obtained biological sample;
Extracting SNP information from the analyzed gene information;
Learning the genetic information through a machine learning technique; And
Reproducing a face image based on the extracted SNP information; Including,
Modeling a correlation through statistical analysis between an attribute value for each part of the face shape obtained by classifying the face shape and SNP information; And
The step of estimating the initial face shape before estimating the face shape; further comprising,
Reproducing the face image,
In the step of modeling the correlation, the correlation between the face shape and the SNP information is modeled, and then the face is estimated using genetic information from the input biological sample,
The step of estimating the initial face shape,
And reproducing the face by merging the shape of the face by applying a weight to an attribute value of each face part to the estimated initial face. The method of claim 5,
Correcting the face image estimated based on the information stated by the witness or the CCTV image. The method of claim 5,
Analyzing the genetic information,
A method for reproducing a face, comprising extracting at least one of information on sex, race, and lineage from the analyzed genetic information. The method of claim 5,
Learning the genetic information through a machine learning technique,
A face reproduction method, characterized in that it learns genetic information using a convolutional neural network (CNN) model.

Images