KR100515085B1 - System and method for acquiring a 3D face digital data from 2D photograph - Google Patents

Abstract

The present invention normalizes the rotational state and size state of a two-dimensional face picture by using two-dimensional feature points extracted from the two-dimensional face picture, and sequentially normalizes the two-dimensional feature points with three-dimensional feature points extracted from a plurality of three-dimensional face shapes. A system and method for acquiring a three-dimensional face shape from a two-dimensional face photograph by comparing with the apparatus, comprising: rotation state normalization means for normalizing a rotation state of the two-dimensional face photograph by using two-dimensional feature points extracted from the two-dimensional face photograph; Size state normalization means for normalizing the size state of a two-dimensional face photograph using the two-dimensional feature points; Storage means for storing a plurality of three-dimensional face shapes; Comparison means for sequentially comparing the two-dimensional feature points of which the rotation state and the size state are normalized with the three-dimensional feature points extracted from the three-dimensional face shape; And production means for selecting a three-dimensional face shape among the stored plurality of three-dimensional face shapes based on the comparison result, and manufacturing a three-dimensional face shape using the selected three-dimensional face shape and the two-dimensional face picture. It can be configured to produce a three-dimensional face shape most similar to a two-dimensional face picture, and there is an effect that can increase the comparative search efficiency in producing a three-dimensional face shape.

Description

System and method for acquiring a 3D face digital data from 2D photograph}

The present invention relates to a system and method for acquiring a three-dimensional face shape from a two-dimensional face picture, and more particularly, a rotation state and a size state of the two-dimensional face picture by using two-dimensional feature points extracted from the two-dimensional face picture. The present invention relates to a system and a method for obtaining a three-dimensional face shape from a two-dimensional face photograph by normalizing and sequentially comparing the normalized two-dimensional feature points with three-dimensional feature points extracted from a plurality of three-dimensional face shapes.

In order to obtain a three-dimensional face shape, several two-dimensional images including a laser scan method, a stereo method, a multi projection method, a moire method, a front side and a side surface are obtained by the optical triangulation method. Methods of synthesizing face pictures and statistically extracting and reproducing three-dimensional face shapes similar to two-dimensional face pictures from a three-dimensional face database have been used.

Laser scanning methods, stereo methods, multi-projection methods, and moiré methods require hardware input devices, and show differences in shooting time and accuracy, but have the advantage of directly acquiring real face data of a person. On the other hand, since the input device must be near the person to be photographed, there is an inconvenience in that a person must go to the input device. In addition, there is a disadvantage that is impossible even in an environment in which a hardware device for acquiring a three-dimensional face shape cannot be installed.

Another method of acquiring three-dimensional face shapes is to synthesize several two-dimensional face photographs including the front and side faces of a person, but it is technically very difficult to restore a three-dimensional face shape of a person from several photos. . Therefore, the 3D face shape read from the database storing the 3D face shape corresponding to the plurality of standard models is compared with the 2D face picture, and the features of the 2D face picture on the 3D face shape similar to the 2D face picture are compared. Research into acquiring and deforming three-dimensional face shapes that are reflected has been actively conducted.

Accordingly, an object of the present invention was created to reflect the above situation, and by normalizing the rotation state of the two-dimensional feature points extracted from the two-dimensional face photograph, the three-dimensional face shape more consistent with the rotation state of the two-dimensional face photograph. It is to provide a system and method for obtaining a three-dimensional face shape from a two-dimensional face photograph to produce a.

In addition, the present invention obtains a three-dimensional face shape from the two-dimensional face picture to produce a three-dimensional face shape more consistent with the size state of the two-dimensional face picture by normalizing the size state of the two-dimensional feature point extracted from the two-dimensional face picture It is another object to provide a system and method.

In addition, the present invention by storing the three-dimensional face shape compared to the two-dimensional face picture in the database in the form of range data (range data), the three-dimensional face shape acquisition system from the two-dimensional face picture to increase the storage efficiency in the database And for other purposes.

In addition, the present invention detects the panning and tilting parameters of the two-dimensional feature points extracted from the two-dimensional face photograph and panning the three-dimensional feature points extracted from the three-dimensional face shape based on the detected panning and tilting parameters. And by tilting, another object is to provide a three-dimensional face shape acquisition system and method from the two-dimensional face picture to produce a three-dimensional side face shape more suited when the two-dimensional face picture is the side.

In addition, the present invention detects the panning and tilting parameters of the two-dimensional feature point, and based on at least one or more preset panning and tilting parameters, the three-dimensional feature point is panned and tilted separately stored in at least one DB (DataBase) Then, one DB is selected from the plurality of stored 3D feature points based on the detected panning and tilting variables, and the 3D feature point is compared with the extracted 2D feature point in the selected DB, thereby matching the 2D face image. It is another object to provide a system and method for acquiring a three-dimensional face shape from a two-dimensional face photograph that enables to produce a three-dimensional face shape more quickly.

In addition, the present invention, by inputting the correction request data for the three-dimensional face shape produced as described above by editing the produced three-dimensional face shape in response to the input correction request data, the request by the input correction request data It is another object to provide a system and method for acquiring a three-dimensional face shape from a two-dimensional face photograph to be reflected.

The present invention for achieving the above object, the rotation state normalization means for normalizing the rotation state of the two-dimensional face picture by using the two-dimensional feature points extracted from the two-dimensional face picture; Size state normalization means for normalizing the size state of a two-dimensional face photograph using the two-dimensional feature points; Storage means for storing a plurality of three-dimensional face shapes; Comparison means for sequentially comparing the two-dimensional feature points of which the rotation state and the size state are normalized with the three-dimensional feature points extracted from the three-dimensional face shape; And production means for selecting a three-dimensional face shape among the stored plurality of three-dimensional face shapes based on the comparison result, and manufacturing a three-dimensional face shape using the selected three-dimensional face shape and the two-dimensional face picture. It provides a three-dimensional face shape acquisition system from a two-dimensional face photograph.

In addition, the present invention solves the above-described problems as a three-dimensional face shape obtaining system from a two-dimensional face photograph, wherein the plurality of three-dimensional face shapes are stored in the form of range data.

In addition, the present invention, the variable detection means for detecting the panning and tilting (left / right and up / down bias) of the two-dimensional face photograph using the extracted two-dimensional feature point; And first panning and tilting means for panning and tilting the extracted three-dimensional feature points based on the detected panning and tilting parameters, wherein the comparing means comprises: two-dimensional normalized rotation and magnitude states A three-dimensional face shape acquisition system from a two-dimensional face photograph, characterized in that the feature points are sequentially compared with the panned and tilted three-dimensional feature points, which solves the above problems.

In addition, the present invention, the second panning and tilting means for panning and tilting the extracted three-dimensional feature point based on at least one preset panning and tilting parameters; And second storage means for dividing and storing the panned and tilted three-dimensional feature points according to the panning and tilting parameters, wherein the comparing means comprises two-dimensional feature points whose rotational state and size state are normalized. The above-mentioned problem is solved by a three-dimensional face shape obtaining system from two-dimensional face photographs, which is sequentially compared with three-dimensional feature points stored in two storage means.

In addition, the present invention, the variable detection means for detecting the panning and tilting (left / right and up / down bias) of the two-dimensional face photograph using the extracted two-dimensional feature point; Second panning and tilting means for panning and tilting the extracted three-dimensional feature points, respectively, based on at least one preset panning and tilting parameter; And second storage means for dividing and storing the panned and tilted three-dimensional feature points according to the panning and tilting parameters, wherein the comparing means comprises two-dimensional feature points whose rotational state and size state are normalized. The above-mentioned problem is solved by a three-dimensional face shape obtaining system from two-dimensional face photographs, which is sequentially compared with three-dimensional feature points stored in two storage means.

In addition, the present invention, the rotation state normalization means, the two-dimensional feature characterized in that to rotate the two-dimensional face photograph so that the straight line connecting two feature points corresponding to both eyes of the extracted two-dimensional feature point is horizontal to the ground The above-mentioned problem is solved as a three-dimensional face shape acquisition system from a face photograph.

In addition, the present invention, the size state normalization means, the two-dimensional face, characterized in that to enlarge or reduce the size of the two-dimensional face photograph so that the distance between any two feature points of the extracted two-dimensional feature point has a specified value The above-mentioned problem is solved as a three-dimensional face shape acquisition system from a photograph.

The present invention may further include editing means for editing the produced three-dimensional face shape in response to the inputted correction request data, when the correction request data is input to the produced three-dimensional face shape. The above-mentioned problem is solved as a three-dimensional face shape acquisition system from a three-dimensional face photograph.

In addition, the present invention for achieving the above object, the first extraction step of extracting a two-dimensional feature point from a two-dimensional face photograph; A rotation state normalization step of normalizing the rotation state of the two-dimensional face photograph by using the extracted two-dimensional feature points; A size state normalization step of normalizing the size state of the two-dimensional face photograph by using the extracted two-dimensional feature points; A second extraction step of extracting a three-dimensional feature point from a plurality of previously stored three-dimensional face shapes; A comparison step of sequentially comparing the two-dimensional feature points of which the rotation state and the size state are normalized with the extracted three-dimensional feature points; And selecting a three-dimensional face shape among a plurality of previously stored three-dimensional face shapes based on the comparison result, and manufacturing a three-dimensional face shape by using the selected three-dimensional face shape and the two-dimensional face picture. It provides a three-dimensional face shape acquisition method from a two-dimensional face photograph made.

In addition, the present invention solves the above-described problems as a method of obtaining a three-dimensional face shape from a two-dimensional face photograph, wherein the plurality of pre-stored three-dimensional face shapes are in the form of range data.

In addition, the present invention, the variable detection step of detecting the panning and tilting (left / right and up / down bias) of the two-dimensional face photograph using the extracted two-dimensional feature point; And a first panning and tilting step of panning and tilting the extracted three-dimensional feature point based on the detected panning and tilting parameters, wherein the comparing step comprises the normalization of the rotational state and the size state. A method for acquiring a three-dimensional face shape from a two-dimensional face photograph, characterized in that the two-dimensional feature points are sequentially compared with the panned and tilted three-dimensional feature points.

The present invention may further include a second panning and tilting step of panning and tilting the extracted three-dimensional feature points, respectively, based on at least one preset panning and tilting parameter; And a second storing step of dividing and storing the panned and tilted three-dimensional feature points according to the panning and tilting parameters, wherein the comparing step includes the two-dimensional feature points whose rotational state and size state are normalized. The above-mentioned problem is solved by a method for acquiring a three-dimensional face shape from a two-dimensional face photograph, which is sequentially compared with the stored three-dimensional feature points.

In addition, the rotation state normalization step, the two-dimensional facial feature, characterized in that to rotate the two-dimensional face photograph so that a straight line connecting two feature points corresponding to both eyes of the extracted two-dimensional feature point is horizontal to the ground As a method of obtaining a three-dimensional face shape from a face photograph, the above-mentioned problem is solved.

In addition, the present invention, the size state normalization step, the two-dimensional face, characterized in that to enlarge or reduce the size of the two-dimensional face photograph so that the distance between any two feature points of the extracted two-dimensional feature point has a specified value As a method of obtaining a three-dimensional face shape from a photograph, the above-mentioned problem is solved.

The present invention may further include an editing step of editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data on the produced three-dimensional face shape. As a method of obtaining a three-dimensional face shape from a two-dimensional face picture, the above-mentioned problem is solved.

In the embodiments to which the present invention is applied, the 3D face image acquisition system is a server computer connected to the Internet network according to the present invention, and the 2D face image is transmitted to the server computer. It is assumed that the subject selectively outputting the correction request data for the three-dimensional face shape produced by the server computer is a client terminal connected to the server computer.

Of course, the client terminal may be connected to the Internet network instead of directly to the server computer as in the present embodiments.

In addition, in the present embodiments, it is proposed that the editing unit provided in the server computer directly performs the editing operation on the three-dimensional face shape requested by the client terminal and the online editing unit connected to the Internet network can be performed. do.

1 is a configuration diagram of an embodiment to which the present invention is applied and receives a two-dimensional face photograph (side and / or front) through an image input apparatus such as a scanner or a camera, and is operated by a user's manipulation. A client terminal 1 for outputting correction request data for the three-dimensional face shape; Stores a plurality of three-dimensional face shapes (projection 3D images) in the form of range data, and three-dimensional face shapes similar to normalized two-dimensional face pictures by normalizing the rotation state and size state of the input two-dimensional face picture. A server computer (2) connected to the Internet network and the client terminal (1) to select and to texture-map the selected three-dimensional face shape and the input two-dimensional face picture to produce a three-dimensional face shape; And an online editing unit 20 that edits the three-dimensional face shape modified by the client terminal 10 online.

The server computer 2 includes: a two-dimensional feature point extracting unit 3 for extracting a two-dimensional feature point from a two-dimensional face photograph transmitted from the client terminal 1; Using the feature points of the two eyes among the extracted 2D feature points, the rotation state of the 2D face photograph in the clockwise or counterclockwise direction is normalized, and when the 2D face photograph is the side, the extracted Rotational state normalization unit for panning expressing the degree of left / right bias in the vertical axis direction and tilting variable expressing the degree of up / down bias in the horizontal axis direction using two-dimensional feature points. Panning and tilting parameter extraction section 4); A size state normalization unit 5 for normalizing the size state of the two-dimensional face photograph by using any two feature points among the two-dimensional feature points whose rotation state is normalized as described above; A projection 3D image DB (DataBase) 13 for storing a three-dimensional face shape (projection 3D image) in the form of range data; A three-dimensional feature point extractor 14 for extracting a three-dimensional feature point from the stored three-dimensional face shape; A conversion unit 15 for converting the extracted three-dimensional feature points into three-dimensional data; A three-dimensional front face feature point DB (8) for storing the converted three-dimensional data; A three-dimensional panning and tilting unit (9) for panning and tilting the converted three-dimensional data based on the detected panning and tilting parameters; A three-dimensional side face feature point DB (7) for storing the panned and tilted three-dimensional data; Comparing the two-dimensional feature points and the three-dimensional feature points stored in the DBs 7 and / or 8, and as a result of the comparison, a three-dimensional face shape (projection 3D image) similar to the two-dimensional feature points is read out from the projection 3D image DB 13. A comparator 6 for temporarily storing the projected 3D image storage unit 12; A texture mapping unit 11 for converting the stored three-dimensional face shape into three-dimensional data and mapping the converted three-dimensional data to a two-dimensional face photograph and texture mapping to produce a three-dimensional face shape; And an editing unit 10 that edits the produced 3D face shape in response to the inputted correction request data when the correction request data is input from the client terminal 1 with respect to the 3D face shape produced as described above. It is composed.

8 illustrates in detail components added to the server computer 2 to perform the editing operation, and includes an interface unit 47 in the editing unit 10 that receives modification request data from the client terminal 1. ; A 3D face correction engine 48 for editing the 3D face shape produced and input by the texture mapping unit 11 in response to the received correction request data; And a modified three-dimensional data storage unit 49 for storing the edited three-dimensional face shape.

Hereinafter, the operation of an embodiment to which the present invention configured as described above is applied will be described in detail in parallel with FIG. 2 which is a flowchart of an embodiment to which the present invention is applied.

First, when the two-dimensional face image of the front (and / or side) is transmitted from the client terminal 1 to the server computer 2 (YES in S201), the two-dimensional feature point extracting unit 3 in the server computer 2 is sent. ) Extracts a two-dimensional feature point from the transmitted two-dimensional face image (S202).

A two-dimensional feature point extraction operation performed by the two-dimensional feature point extractor 3 will be described with reference to the flowchart of FIG. 5A.

When the two-dimensional face picture is transmitted (S501), the two-dimensional feature point extractor 3 divides the inside of the face and the outside of the background by the color information of the transmitted face picture (S502). If the color information is flesh color, it corresponds to the inner part of the face, and if it is not flesh color, it corresponds to the outside part of the background.

The two-dimensional feature extraction unit 3 fills in information on each part of the face (both eyes, eyebrows, the lower part of the nose, and the mouth) generated in the inner region of the color of the face image separated by the flesh color. After completely separating only the face (S503), two-dimensional projection (2D projection) in the horizontal and vertical directions with respect to the boundary portions of the face parts to obtain 2D projection data and interpret the location information of each face part (S504). In the 2D projection data obtained as described above, the values of the 2D projection data projected from the eyebrows, the eyes and the nose, and the lower part of the mouth are lower than those of the skin parts, rather than the skin areas. Find it.

The two-dimensional feature point extractor 3 extracts the feature points of each of the face parts using the positions of the respective parts of the 2D projected face in this way (S505), as shown in FIG. 6A. The centers c and d, the nose end b for the nose, and the center a of the mouth for the mouth are extracted as respective feature points. The above feature points are used for the overall face comparison of the face, and for comparison with the partial face obtained in the three-dimensional face shape, the eye top / bottom / left / right feature points and mouth to obtain the width and height of the eye, and the width of the mouth Extract the left and right feature points of.

In addition, the front and side pictures transmitted in step S501 extract the outline of the entire face and the jaw outline for graph matching, and in the case of the side picture in which the outline of the nose appears, the outline of the nose is also extracted ( S506).

After the two-dimensional feature point is extracted as described above, the rotation state normalization unit 4 normalizes the rotation state of the two-dimensional face photograph using the extracted two-dimensional feature point (S203), for example, two of the two-dimensional feature points. The two-dimensional face photograph is rotated so that a straight line connecting two feature points (c and d in FIG. 6A) corresponding to the eyes is horizontal.

In this case, only when the transmitted 2D face photograph is not the front face (No in S204), the panning and tilting variable detection unit 4 may match the panning parameter corresponding to the left / right offset of the 2D face photograph. A tilting variable corresponding to a degree / down bias is detected (S205). Means for detecting the degree of panning bias include comparing the areas of the left and right small triangles cba and triangle dba made from the feature points of FIG. 6a, comparing the angles of the small triangle vertices, or vertices b or a at small triangle vertices c and d. A method of comparing vectors directed in directions is used. As a means for detecting the tilting bias, the tilting variable may be extracted according to the degree of the position of the feature point a with respect to the straight line ef connecting the feature points e and f representing the left and right ends of the mouth. For example, when the position of the feature point a lies on a straight line, the tilting variable is recognized as the front corresponding to '0'.

The size state normalizing unit 5 normalizes the size state of the two-dimensional face photograph by using any two feature points among the two-dimensional feature points (S206), for example, two feature points (c and d) corresponding to two eyes. Magnify or reduce the two-dimensional face image until the length of the straight line is equal to the specified length (L).

On the other hand, when the two-dimensional face image is transmitted to the server computer 2 (YES in S201), the three-dimensional feature point extractor 14 extracts the three-dimensional face shape (projection 3D image) from the projection 3D image DB 13. The 3D feature point is extracted from the 3D face shape that is read out while being sequentially read (S207). At this time, the length of a straight line connecting two feature points corresponding to two eyes among the extracted 3D feature points is always the same as the designated length L in the size state normalization operation for the 2D face photograph.

A three-dimensional feature point extraction operation performed by the three-dimensional feature point extractor 14 will be described with reference to the flowchart of FIG. 5B.

The three-dimensional feature point extractor 14 extracts a pixel having the maximum range data from the three-dimensional face image to be read out when the three-dimensional face shape in the range data form is sequentially read from the projection 3D image DB 13 (S511). (S512).

FIG. 7 is a diagram showing an embodiment of a projection 3D image, which is data constituting the projection 3D images DB 13 and 43. The projection 3D image is a method of recording three-dimensional shape information on the projection surface. The projection 3D image itself is sheet data and data having three-dimensional stereoscopic information. However, the data storage method is different from the three-dimensional data storage method of the general meaning.

In the general three-dimensional data storage method, a vertex value for an arbitrary point in space and an index value for generating each plane for each point in the space are stored together. Generally, since the shape of the minimum vertex that creates one face is a triangle, one vertex value has x, y, and z values in Cartesian coordinates, and one index value corresponds to vertex 1 and vertex 2 corresponding to each vertex. , The value of vertex 3 is obtained.

However, in the case of projection 3D data, the information on the projected surface is in accordance with the information type already promised in advance, and only the distance information with the subject corresponding to each point in the projection surface is stored. It will be stored as a contrast value between 0 and 255, or more precisely. The data projecting the subject onto a specific surface is also called range data or depth data, and the accuracy of the projected distance information is used to allocate more bytes for data processing. It can scale to infinity in this way.

The projection 3D image data of FIG. 7 is a result of recording only the stereoscopic information of the appearance in the projection direction by projecting a human face onto a plane in one direction of the front, and thus corresponding to 3D, 2D and 1 / 2D, which are intermediate between 2D and 3D. Also called 2.5D. However, it is possible to produce the entire human head as a projection 3D image by recording the appearance information of the face on the cylindrical surface of the cylindrical coordinate system while rotating the outline of the face 360 °, and project the projection 3D image of the cylindrical coordinate system to the projection 3D image DB (13 ) And the same means of the present invention can be applied to produce a three-dimensional face shape expressing the entire 360 ° around in more detail.

Therefore, the 3D feature point extractor 14 extracts a pixel having the maximum contrast value (assuming that the contrast value is the brightest for convenience) as the most protruding portion in the face region of the range data (S512). The extracted pixel is selected as the feature point B corresponding to the tip of the nose (S513).

Subsequently, the three-dimensional feature extraction unit 14 finds a feature point corresponding to the weight that is a sharply entered portion below the tip of the nose (S514), and when the weight below the tip of the nose is found, there is a slightly convex upper lip in the downward direction. , The lower lip is convex in the direction of the upper lip, finds the left and right lip end points where the upper lip and the lower lip meet, and finds a feature point (A) corresponding to the left / right center of the centerline where the upper lip and the lower lip meet each other. Finding a feature point corresponding to the mouth (S515), searching for the stomach at the tip of the nose finds a feature point corresponding to the brow, which is gradually lowered and then raised again (S516), both eyes present at the lower points corresponding to the pupils on both sides of the brow. Next, find the feature points representing the up / down / left / right endpoints from the center of the pupil and the two feature points (C and D) corresponding to the centers of both eyes. The (S517).

The extracted feature points A to D are as shown in FIG. 6B, and the conversion unit 15 converts the extracted feature points into 3D data (S208), and then transfers the two-dimensional face photographs previously transmitted. In the case of the front face (YES in S209), the converted 3D data is stored in the 3D front face feature point DB (S213). From the projection 3D image data or the 3D face shape representing the 3D face shape at the same time, Extract the outline information of the face and the outline of the jaw from the data converted into the projected 3D image and save it in the database.After rotating the 3D face shape under the panning and tilting conditions of the face, It also extracts the outline of the nose.

On the contrary, when the transmitted two-dimensional face photograph is not a front side (“No” in S209) and the panning and tilting variables are detected and input by the panning and tilting variable detector 4 (“Yes” in S210). , The three-dimensional panning and tilting unit 9 pans and tilts the three-dimensional feature points converted into three-dimensional data based on the input panning and tilting variables (S211) and stores them in the three-dimensional side face feature points DB 7. (S212).

On the other hand, the comparison unit 6 first uses a two-step comparison method of comparing the overall geometric feature (geometric feature matching) and then the feature of the facial features (local feature analysis). In detail, among the three-dimensional front face (or side face) feature points DB 8 or 7, the three-dimensional feature points A stored in the DB 8 or 7 based on whether the face photograph transmitted in step S201 is the front or the side. -D) and the two-dimensional feature points a to d whose rotational state and size state are normalized are sequentially compared (S214), and in the next step, the left / right width, the upper / lower height of each eye, and the width of the mouth are compared. Compare and compare the outline of the face with the outline of the jaw and the outline of the nose in the side face with a graph matching method. As a result of the comparison, after selecting a three-dimensional feature point similar to the two-dimensional feature point (S215), the projection 3D image corresponding to the selected three-dimensional feature point is read out from the projection 3D image DB 13 to resemble the projection 3D image storage unit 12. Store one or more, depending on the degree.

Subsequently, the texture mapping unit 11 receives a request for texture mapping and three-dimensional face shape selection from the client terminal 1, and then the three-dimensional face shape stored in the projection 3D image storage unit 12, i. After converting to the dimensional data (S216), to produce a three-dimensional face shape by mapping the texture and the two-dimensional face image transmitted in step S201 (S217), if the produced three-dimensional face shape is not satisfactory projection 3D image Steps S216 and S217 are repeatedly performed by selecting a similar three-dimensional feature point (YES in S218) by selecting and producing a new projection 3D image stored in the storage unit 12.

After this, if the correction request data for the three-dimensional face shape produced as described above from the client terminal 1 is input to the interface unit 47 of the editing unit 10 as shown in FIG. 8 (S219). The 3D face correction engine 48 edits the produced 3D face shape in response to the inputted correction request data (S220) and stores the modified 3D data storage 49 in the modified 3D data storage unit 49. In addition, when inputting (YES in S221), step S220 and the storage operation to the modified three-dimensional data storage unit 49 are repeatedly performed.

On the contrary, when the correction request data is input through the interface unit 47 of the editing unit 10, the 3D face correction engine 48 does not perform the editing operation of the 3D face shape. Of course, it is possible to transmit the input correction request data to the online editing unit 20 connected to the Internet network and to perform the online editing operation of the three-dimensional face shape by the online editing unit 20.

3 is a block diagram of another embodiment to which the present invention is applied. In addition to the same components as those of FIG. 1, the three-dimensional side face feature point DB 37 is divided into eleven DBs 37a to 11 divided by eleven panning and tilting parameters. It is characterized by consisting of 37k).

Therefore, in the embodiment of FIG. 1, the three-dimensional panning and tilting unit 9 pans and tilts the three-dimensional feature points based on the panning and tilting variables detected by the panning and tilting variable detection unit 4. Although the feature point DB 7 stores this, in the present embodiment, the 3D panning and tilting unit 39 pans and tilts the 3D feature points in advance by 11 panning and tilting variables, respectively, and the 3D side face feature point DB 37 ) Stores them separately.

Therefore, in the exemplary embodiment of FIG. 1, when the face image transmitted is the side, the comparator 6 reads the 3D feature point from the 3D side face feature point DB 7 and compares it with the 2D feature point extracted earlier. In the present embodiment, when the transmitted face photograph is a side, a DB corresponding to the panning and tilting variables input from the panning and tilting variable detector 34 among the 3D side face feature points DBs 37a to 37k. There is a difference between selecting and comparing the three-dimensional feature point read from the selected DB with the two-dimensional feature point extracted earlier.

Hereinafter, the operation of another embodiment to which the present invention configured as described above will be described in parallel with FIGS. 4A to 4C, which are flowcharts of another embodiment to which the present invention is applied, but will only be described in detail with respect to portions different from the above-described embodiment. Shall be.

First, as shown in FIG. 4A, in the present embodiment, the 3D feature point extractor 44 may construct the feature points DB 37 and 38 in advance before the face photograph of the customer is input, and from the projection 3D image DB 43. A 3D face shape (projection 3D image) is read and a 3D feature point is extracted therefrom (S401), and the conversion unit 45 converts the extracted 3D feature point into 3D data (S402).

In order to produce the three-dimensional side face feature point DB 37 ("side face feature point DB" of S403), the three-dimensional panning and tilting unit 39 sets a plurality of preset three-dimensional feature points (11 in this embodiment). It is assumed to be individual.) After panning and tilting based on the panning and tilting variables (S404), the three-dimensional side face feature point DB 37 is separately stored in the corresponding DB among eleven DBs 37a to 37k.

At the same time, in order to produce the front face feature point DB 38 ("front face feature point DB" of S403), the converted three-dimensional data is stored in the three-dimensional front face feature point DB 38 (S406).

4B and 4C, when the two-dimensional front (and / or side) face photograph is transmitted from the client terminal 31 (S407), the two-dimensional feature point extraction unit 33 in the server computer 32 may be used. The two-dimensional feature point is extracted from the transmitted two-dimensional face image (S408), and the rotation state normalization unit 34 normalizes the rotation state of the two-dimensional face image by using the extracted two-dimensional feature point (S409).

In this case, when the transmitted two-dimensional face image is the side (No in S410), the panning and tilting variable detector 34 detects the panning and tilting variables of the transmitted two-dimensional face image (S411). Subsequently, the size state normalization unit 35 normalizes the size state of the two-dimensional face photograph by using the extracted two-dimensional feature points (S412).

On the other hand, after the panning and tilting variable is detected by the panning and tilting variable detection unit 34 in step S411 (YES in S413), the comparison unit 36 is a center of the three-dimensional side face feature points DB 37a to 37k. Select a DB storing the 3D feature points corresponding to the detected panning and tilting variables, and sequentially store the 3D feature points A to D stored in the selected DB with the 2D feature points a to d extracted in step S402. (S414), select a three-dimensional feature point similar to the two-dimensional feature point (S415), and read the projected 3D image corresponding to the selected three-dimensional feature point from the projected 3D image DB 43 to project the projected 3D image. The storage unit 42 stores it.

Subsequently, upon receiving a request for texture mapping and three-dimensional face shape selection from the client terminal 31, the texture mapping unit 41 displays the three-dimensional face shape stored in the projection 3D image storage unit 42, that is, the projection 3D image. After the conversion to the dimensional data (S416), the three-dimensional face shape is produced by texture mapping with the two-dimensional face image transmitted in step S407 (S417), and if the produced three-dimensional face shape is not satisfactory, the projection 3D image Steps S416 and S417 are repeatedly performed by selecting a new three-dimensional (3D) image stored in the storage unit 42 and further selecting similar three-dimensional feature points (YES in S418).

After doing this, when the correction request data for the three-dimensional face shape produced as described above is input from the client terminal 31 to the interface unit 47 of the editing unit 40 as shown in FIG. 8 (S419). The 3D face correction engine 48 edits the produced 3D face shape in response to the input correction request data (S420) and stores the modified 3D data storage unit 49 in the modified 3D data storage unit 49. In addition, when an input (YES in S421) is performed, the operation S420 and the storage operation in the modified 3D data storage unit 49 are repeatedly performed.

In addition, when the correction request data is input through the interface unit 47 of the editing unit 40, the 3D face correction engine 48 does not perform the editing operation of the 3D face shape, and inputs the 3D face shape and the produced 3D face shape. The modified request data may be transmitted to the online editing unit 50 connected to the internet network, and the online editing unit 50 may perform the online editing operation of the three-dimensional face shape.

The present invention is not limited to the embodiments described above, and various modifications and changes can be made by those skilled in the art, which are included in the spirit and scope of the present invention as defined in the appended claims.

As described above, the present invention has the effect of producing a three-dimensional face shape most similar to a two-dimensional face photograph.

In addition, the present invention has the effect of improving the comparison search efficiency in producing a three-dimensional face shape by comparing the two-dimensional face photograph and the three-dimensional face shape secured in advance.

1 is a configuration of an embodiment to which the present invention is applied,

2a and 2b is a flow chart of an embodiment to which the present invention is applied,

3 is a configuration diagram of another embodiment to which the present invention is applied;

4a to 4c is a flow chart of another embodiment to which the present invention is applied,

5A is a flowchart of extracting two-dimensional feature points from a two-dimensional face photograph,

5B is a flowchart for extracting three-dimensional feature points from a three-dimensional face shape.

FIG. 6A illustrates a two-dimensional face photograph and two-dimensional feature points extracted therefrom,

FIG. 6B is a view illustrating a three-dimensional front face shape and three-dimensional feature points extracted therefrom,

FIG. 6C illustrates a three-dimensional side face shape and three-dimensional feature points extracted therefrom.

7 is a view showing an embodiment of a projection 3D image,

FIG. 8 illustrates in detail the components added to the server computers 2 and 32 to perform the editing operation by the request of the client terminals 1 and 31 in the configuration of FIGS. 1 and 3.

<Explanation of symbols for the main parts of the drawings>

1,31: client terminal 2,32: server computer

3,33: 2D feature point extraction unit

4,34: rotation state normalization unit (panning and tilting variable detection unit)

5,35: Size state normalization unit 6,36: Comparison unit

7,37: 3D side face feature point DB

8,38: 3D front face feature point DB

9,39: 3D panning and tilting unit 10,40: Editing unit

11,41 texture mapping unit 12,42 projection 3D image storage unit

13,43: Projection 3D image DB 14,44: 3D feature point extraction unit

15,45: converter 20,50: online editor

47: interface unit 48: 3D face correction engine

49: modified three-dimensional data storage unit

Claims (66)

delete Rotational state normalization means for normalizing the rotational state of the two-dimensional face photograph by using the two-dimensional feature points extracted from the two-dimensional face photograph; Size state normalization means for normalizing the size state of a two-dimensional face photograph using the two-dimensional feature points; Storage means for storing a plurality of three-dimensional face shapes; Comparison means for sequentially comparing the two-dimensional feature points of which the rotation state and the size state are normalized with the three-dimensional feature points extracted from the three-dimensional face shape; And A production means for selecting one three-dimensional face shape among the stored plurality of three-dimensional face shapes based on the comparison result, and manufacturing a three-dimensional face shape using the selected three-dimensional face shape and the two-dimensional face picture; , The three-dimensional face shape obtaining system from a two-dimensional face picture, characterized in that the plurality of three-dimensional face shape is stored in the form of range data (range data). The method of claim 2, Variable detecting means for detecting a panning and tilting (left / right and up / down bias) variable of a two-dimensional face photograph by using the extracted two-dimensional feature points; And First panning and tilting means for panning and tilting the extracted three-dimensional feature points based on the detected panning and tilting parameters, And the comparing means sequentially compares the two-dimensional feature points of which the rotation state and the size state are normalized with the panned and tilted three-dimensional feature points. The method of claim 3, wherein Second panning and tilting means for panning and tilting the extracted three-dimensional feature points based on at least one preset panning and tilting parameter; And And second storage means for classifying and storing the panned and tilted three-dimensional feature points according to the panning and tilting parameters. The comparison means, And a two-dimensional feature point of which the rotation state and the size state are normalized are sequentially compared with the three-dimensional feature points stored in the second storage means. The method of claim 2, Variable detecting means for detecting a panning and tilting (left / right and up / down bias) variable of a two-dimensional face photograph by using the extracted two-dimensional feature points; Second panning and tilting means for panning and tilting the extracted three-dimensional feature points, respectively, based on at least one preset panning and tilting parameter; And And second storage means for classifying and storing the panned and tilted three-dimensional feature points according to the panning and tilting parameters. The comparison means, And a two-dimensional feature point of which the rotation state and the size state are normalized are sequentially compared with the three-dimensional feature points stored in the second storage means. The method of claim 2, wherein the rotation state normalization means, And a two-dimensional face image obtained from the two-dimensional face photograph, wherein the two-dimensional face photograph is rotated so that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 3, wherein the rotation state normalization means, And a two-dimensional face image obtained from the two-dimensional face photograph, wherein the two-dimensional face photograph is rotated so that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 4, wherein the rotation state normalization means, And a two-dimensional face image obtained from the two-dimensional face photograph, wherein the two-dimensional face photograph is rotated so that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 5, wherein the rotation state normalization means, And a two-dimensional face image obtained from the two-dimensional face photograph, wherein the two-dimensional face photograph is rotated so that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 2, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 3, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 4, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 5, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 6, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 7, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 8, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 9, wherein the size state normalization means, And a size of the two-dimensional face image is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 2, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 3, wherein And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 4, wherein And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 5, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 6, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 7, wherein And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 8, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 9, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 10, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 11, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 12, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 13, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 14, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 15, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 16, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. The method of claim 17, And editing means for editing the produced three-dimensional face shape in response to the inputted correction request data when inputting correction request data for the three-dimensional face shape. Face shape acquisition system. delete A first extraction step of extracting a two-dimensional feature point from the two-dimensional face photograph; A rotation state normalization step of normalizing the rotation state of the two-dimensional face photograph by using the extracted two-dimensional feature points; A size state normalization step of normalizing the size state of the two-dimensional face photograph by using the extracted two-dimensional feature points; A second extraction step of extracting a three-dimensional feature point from a plurality of previously stored three-dimensional face shapes; A comparison step of sequentially comparing the two-dimensional feature points of which the rotation state and the size state are normalized with the extracted three-dimensional feature points; And Selecting a three-dimensional face shape among a plurality of three-dimensional face shapes previously stored based on the comparison result, and manufacturing a three-dimensional face shape using the selected three-dimensional face shape and the two-dimensional face picture; , A plurality of pre-stored three-dimensional face shape is a method of obtaining a three-dimensional face shape from a two-dimensional face picture, characterized in that the range data (range data) form. The method of claim 35, wherein A variable detecting step of detecting a panning and tilting (left / right and up / down bias) variable of a two-dimensional face image by using the extracted two-dimensional feature points; And A first panning and tilting step of panning and tilting the extracted three-dimensional feature points based on the detected panning and tilting parameters, And the comparing step sequentially compares two-dimensional feature points whose rotational state and size state are normalized with the three-dimensional feature points panned and tilted. The method of claim 36, A second panning and tilting step of panning and tilting the extracted three-dimensional feature points, respectively, based on at least one preset panning and tilting variable; And And storing the panned and tilted three-dimensional feature points separately according to the panning and tilting parameters. The comparing step, And a two-dimensional feature point of which the rotation state and the size state are normalized are sequentially compared with the three-dimensional feature points stored separately. The method of claim 35, wherein A variable detecting step of detecting a panning and tilting (left / right and up / down bias) variable of a two-dimensional face image by using the extracted two-dimensional feature points; A second panning and tilting step of panning and tilting the extracted three-dimensional feature points, respectively, based on at least one preset panning and tilting variable; And And storing the panned and tilted three-dimensional feature points separately according to the panning and tilting parameters. The comparing step, And a two-dimensional feature point of which the rotation state and the size state are normalized are sequentially compared with the three-dimensional feature points stored separately. The method of claim 35, wherein the rotating state normalization step, And a two-dimensional face image obtained by rotating a two-dimensional face image such that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 36, wherein the rotation state normalization step, And a two-dimensional face image obtained by rotating a two-dimensional face image such that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 37, wherein the rotation state normalization step, And a two-dimensional face image obtained by rotating a two-dimensional face image such that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 38, wherein the rotation state normalization step, And a two-dimensional face image obtained by rotating a two-dimensional face image such that a straight line connecting two feature points corresponding to both eyes among the extracted two-dimensional feature points is horizontal with the ground. The method of claim 35, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 36, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 37, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 38, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. 40. The method of claim 39, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 40, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. 42. The method of claim 41, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. 43. The method of claim 42, wherein the size state normalization step, 3. The method of claim 3, wherein the size of the two-dimensional face photograph is enlarged or reduced so that the distance between any two feature points among the extracted two-dimensional feature points has a specified value. The method of claim 35, wherein When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 36, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 37, wherein When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 38, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 39, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 40, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. 42. The method of claim 41 wherein When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 42, wherein When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 43, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 44, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 45, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 46, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 47, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 48, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. The method of claim 49, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape. 51. The method of claim 50, When the correction request data is input to the produced three-dimensional face shape, the editing step of editing the produced three-dimensional face shape in response to the input correction request data further comprises three from the two-dimensional face picture, characterized in that How to obtain dimensional face shape.