KR20090065965A - 3d image model generation method and apparatus, image recognition method and apparatus using the same and recording medium storing program for performing the method thereof - Google Patents

Abstract

A method and a device for generating a 3D(Dimensional) image model, an image recognition method and device using the same, and a recording medium storing a program thereof are provided to improve processing speed and correctness needed for generating the 3D image model by converting an image on 2D space. A 3D image feature selector(20) selects 3D image features of each 3D scan image. A 2D space projector(30) projects each 3D scanning image to 2D space. An image modifier(40) determines one of the projected 2D images from a reference image. The image modifier modifies the 2D images excluding the reference image according to a shape of the reference image. A 3D image recovering unit(50) extracts 2D image features of each modified image and recovers the 3D images by using the 2D image features. A model generator(60) generates a 3D image model by using the 3D images.

Description

3D image model generation method and apparatus, image recognition method and apparatus using the same and recording medium storing program for performing the method according to

The present invention relates to a method for generating a 3D image model and an image recognition method using a 3D image model used for 3D image reconstruction, and more particularly, to a biometric system for reconstructing an arbitrary face image and recognizing and verifying an individual's identity. The present invention relates to a 3D image model generation method and a face recognition method and apparatus used in the present invention.

The 3D face model has the advantage of reconstructing the 3D face shape from the 2D image of a specific person and enabling the face recognition to be robust in the pose and lighting of the face. have. The three-dimensional face model technique is replacing the existing two-dimensional face recognition and other image processing methods.

In order to generate a three-dimensional face model, it is necessary to solve the problem of dense correspondence that obtains a set of one-to-one corresponding dense facial feature points from a plurality of three-dimensional face scan data.

There are two existing methods to solve this problem. One is to find out the color information of each feature point by matching the feature points in the form of a predefined complex 3D face model to the 3D scan data, and the other is to extract the required feature points from the 3D scan data. There is a method of generating shape information and color information. In this case, the feature points should be a common feature point that has a one-to-one correspondence to all the scan data. In general, the feature points range from 10,000 to 20,000, so it is important to efficiently extract the face feature points.

As a conventional method for extracting a plurality of feature points, there is a method of automatically extracting 10,000 to 20,000 dense face feature points between three-dimensional face scan data using optical flow. However, since the method using light flow is a method of matching image blocks of a certain size, the accuracy of matching is very low in case of 3D scan data having different shapes and colors of the face, and it is sensitive to the initial position where the matching starts. For example, when the size of a block is large, there is a problem in that a large amount of calculation is required for matching.

In addition, according to the conventional face recognition method, the recognizer is trained using the 3D facial feature points and the color information of the restored 3D face, and the process of restoring the 3D face with respect to the 2D image input for face recognition is performed. It is necessary to obtain feature points and color information through this. In this case, there is a disadvantage in that the face recognition speed is very slow due to a large amount of computation in the 3D face restoration process.

As a patent document, Korean Patent No. 360487 discloses a method of mapping a texture from a two-dimensional face image to a three-dimensional face model. In this case, a three-dimensional face image close to the two-dimensional face using predefined predefined control points. Since there is a limit to the improvement of the recognition efficiency.

In view of the above-described limitations of the prior art, the present invention reduces the amount of computation associated with the generation of a 3D image model by performing a transformation on a 2D image that corresponds to a selected number of 3D image feature points from 3D scan data. It is an object of the present invention to provide a method and apparatus for generating a 3D image model that can improve the accuracy of face recognition.

In addition, the present invention is to create a three-dimensional image from the two-dimensional training image by using a three-dimensional image model by using the two-dimensional images obtained by changing the pose and lighting as data for image recognition, the two-dimensional to be recognized An object of the present invention is to provide a method and apparatus for recognizing an image having robustness to pose and lighting without reconstructing an input image into a 3D image.

According to an aspect of the present invention, there is provided a method of generating a 3D image model, the method including: selecting 3D image feature points according to 3D scan images; Projecting each of the three-dimensional scanned images into a two-dimensional space; Determining one image from the projected two-dimensional images as a reference image, and deforming the two-dimensional images other than the reference image to correspond to the shape of the reference image; Extracting two-dimensional image feature points according to each of the modified images, and reconstructing three-dimensional images using the extracted two-dimensional image feature points; And generating a 3D image model using the reconstructed 3D images.

According to another aspect of the present invention, there is provided a apparatus for generating a 3D image model, comprising: a 3D image feature point selector configured to select 3D image feature points according to 3D scan images; A two-dimensional space projector configured to project each of the three-dimensional scanned images into a two-dimensional space; An image transformation unit configured to determine one image from the projected two-dimensional images as a reference image, and to modify the remaining two-dimensional images other than the reference image to correspond to the shape of the reference image; A 3D image reconstructing unit which extracts 2D image feature points according to each of the modified images, and reconstructs 3D images using the extracted 2D image feature points; And a model generator for generating a 3D image model using the reconstructed 3D images.

According to another aspect of the present invention, there is provided a method of recognizing an image, selecting three-dimensional image feature points corresponding to each of a three-dimensional scanned image, and projecting each of the three-dimensional scanned image in a two-dimensional space; Determining one image from the projected two-dimensional images as a reference image, and deforming the two-dimensional images other than the reference image to correspond to the shape of the reference image; Extracting two-dimensional image feature points corresponding to each of the modified images, and reconstructing a three-dimensional image by using the extracted two-dimensional image feature points; Generating a 3D image model using the reconstructed 3D images; Receiving 2D training images and generating 3D composite images predicted from the training images using the 3D image model; Extracting a feature vector according to each training image from the generated 3D composite images; And receiving a 2D image of the input image to be recognized, extracting a feature vector from the input 2D image, and using the similarity between the feature vector according to the input image and the feature vectors according to the training image. Calculating a recognition result for the image.

According to another aspect of the present invention, there is provided an apparatus for recognizing an image, comprising: a 3D image feature point selector which selects 3D image feature points according to 3D scan images; A two-dimensional space projector configured to project each of the three-dimensional scanned images into a two-dimensional space; An image transformation unit configured to determine one image from the projected two-dimensional images as a reference image, and to modify the remaining two-dimensional images other than the reference image to correspond to the shape of the reference image; A 3D image reconstructing unit which extracts 2D image feature points according to each of the modified images, and reconstructs a 3D image using the extracted 2D image feature points, respectively; A model generator for generating a 3D image model using the reconstructed 3D images; A 3D image generator which receives 2D training images and generates 3D composite images predicted from the training images using the 3D image model; A feature vector extracting unit extracting a feature vector according to each training image from the generated 3D composite images; An image input unit configured to receive a 2D image of the input image to be recognized; And an image determiner configured to extract a feature vector for the input image and calculate a recognition result for the input image using similarity between the feature vector according to the input image and the feature vectors according to the training image.

According to the method and apparatus for generating a 3D image model of the present invention, a 3D image model is performed by performing transformation on an image in 2D to correspond to a few feature points selected from 3D scan data and 3D image feature points. It can improve the processing speed and accuracy required to generate the.

In addition, according to the image recognition method of the present invention, after generating a three-dimensional image by using a three-dimensional image model of the two-dimensional training image to change the pose and lighting to obtain a variety of two-dimensional image as data for image recognition By utilizing the two-dimensional input image to be recognized, the image recognition that is resistant to pose and lighting is possible without restoring the three-dimensional image. In addition, processing time required for reconstructing the 2D input image into the 3D image can be reduced, and the system can be simplified.

Hereinafter, a method and apparatus for generating a 3D image model, an image recognition method and apparatus using the same, and a recording medium on which a program for performing the methods are recorded will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an apparatus for generating a 3D image model according to an embodiment of the present invention. The apparatus for generating a 3D image model 1 shown in FIG. 1 includes a 3D scan image input unit 10, a 3D image feature point selector 20, a 2D space projector 30, an image converter 40, The 3D image reconstructor 50 and the model generator 60 are included.

The 3D scan image input unit 10 receives 3D scan image information and stores the input information. The 3D scan image information includes pixel-specific location information and color information obtained from different people. For example, the three-dimensional face scan information is composed of 50,000 to 100,000 three-dimensional position information and color information accordingly. Face scan information requires three-dimensional scan face information for as many people's faces as possible in order to generate a three-dimensional face model for everyone in a group.

The 3D image feature point selector 20 selects a 3D image feature point for each 3D scan image according to a predetermined criterion. The 3D image feature point selector 20 may extract only a few feature points that can effectively specify a face image using known algorithms. In particular, the 3D image feature point selector 20 may extract a predetermined number of feature points associated with major face feature positions such as eyes, eyebrows, nose, and mouth from the 3D scan face image using the OpenGL picking algorithm. Since the selection of the three-dimensional image feature point only needs to select a few feature points that effectively represent the characteristics of the image, in this case, it may be implemented to select the feature point of the three-dimensional image through manual input.

Also, the 3D image feature point selector 20 may further include a mesh generator (not shown) for generating triangular meshes according to the selected predetermined number of feature points. The mesh generator may determine triangular meshes according to the extracted feature points through a Delaunay triangulation technique.

2A and 2B are reference diagrams illustrating an example of extracting a predetermined number of feature points from a 3D face scan image and generating a triangle mesh. 2A represents the index of a total of 115 feature points related to the main elements of the face, namely eyes, eyebrows, nose, mouth, jaw line, forehead, and the like. 2B shows a triangular mesh according to 115 feature points using the Delaunay triangulation technique.

The two-dimensional space projecting unit 30 projects the three-dimensional scan image in two-dimensional space having two axes of rotation angle and height rotating about a predetermined central axis.

3 is a reference diagram illustrating an example in which a 3D face scan image is projected onto a 2D cylinder space. In FIG. 3, a is an image representing color information according to each pixel projected on a two-dimensional cylinder space, and b is an image showing a rotation radius when the central axis of the face is referenced.

The image deforming unit 40 determines one image from the projected two-dimensional images as a reference image, and deforms the other two-dimensional images except the reference image to correspond to the reference image. The image transform unit 40 includes a reference image determiner 42, a mask region determiner 44, and an affine converter 46.

The reference image determiner 42 determines one image from the projected two-dimensional images as the reference image. The mask area determiner 44 determines a mask area according to feature points located at the outer side among the feature points of the reference image. The affine transform unit 46 performs an affine transform based on the feature points or all the feature points forming the determined mask area. As a result of the affine transformation, the feature point positions of the remaining images except the reference image are converted to correspond to the reference image.

4 is a reference diagram illustrating an example of affine transforming an image in a two-dimensional cylinder space according to an embodiment of the present invention. In FIG. 4, the 102 image is a reference image selected from two-dimensional images projected on a two-dimensional cylinder space, and the 104 image is an image having different shape and color information from the reference image 102. The 106 image represents a mask area, and the boundary of the mask area may be specified by connecting the outermost feature points among the 115 feature points of the reference image. The 108 images are transformed by affine transformation based on the 104 reference image. For example, when the 104 image is affine transformed based on the mask area of the 102 image, the pixels of all the face images may be transformed to correspond to each other through the reference face image and the affine transformation.

In the present embodiment, the images output from the image transformation unit are modified to correspond to the feature points of the reference image through affine transformation of each feature point. The image transformation unit may assign an index according to each pixel in the affine transformation process. Since all pixels may correspond to each other according to the index per pixel, a process for matching feature points such as image registration is not necessary later. .

The 3D image reconstructor 50 extracts dense feature points of the 2D image according to each of the modified images, and reconstructs the 3D image using the extracted dense feature points. The 3D image reconstructor 50 includes a dense feature point extractor 52, a mesh generator 54, an inverse affine converter 56, and a reverse projector 58.

The dense feature extraction unit 52 extracts all coordinates or pixels included in the mask area as feature points of the 2D image.

In order to obtain a three-dimensional face model, a compact face feature set corresponding to all the scan data must be obtained. The dense feature extractor 52 may extract a very dense facial feature point through linear interpolation of a triangular mesh composed of 115 limited number of feature points and the feature points.

The mesh generator 54 generates triangular meshes connecting the dense feature points extracted by the dense feature point extractor 52. In particular, the mesh generator 54 may generate a triangular mesh composed of dense feature points using a Delaunay triangulation technique. Once the triangular mesh is determined, when the 3D image model is displayed on the screen, the image may be represented in the form of a plane according to the triangular mesh without being represented as a dot.

In particular, in the case of a 3D face image, the front face of the face must be expressed in the original color and the back face must be black so that the expression of the 3D face is not distorted. When the triangle mesh is defined in a constant direction (clockwise or counterclockwise), You can determine whether the front or back of each mesh can prevent distortion. In addition, the triangular mesh may be used to apply a weight to each pixel defined in the 3D image model in the 3D image reconstruction process.

For example, if the two-dimensional face image is a side image instead of the front face, there may be pixels defined in the three-dimensional face model but invisible in the two-dimensional face image, in which case low weights are given to the invisible pixels. 3D face reconstruction can be done more precisely by giving higher weights to the visible pixels. Here, the triangle mesh is used to calculate the weight. In this case, the weight may be lowered.

The inverse affine transformation unit 56 converts the images in which the triangular mesh is generated into an image existing in the two-dimensional cylinder space through the inverse affine transformation. In particular, the inverse fishnet transformation unit 56 may perform inverse fishnet transformation according to Equation 1 below.

[Equation 1]

Where m is the horizontal coordinate in cylinder space for the affine transformed image, n is the vertical coordinate in cylinder space for the affine transformed image, u is the horizontal coordinate in cylinder space for the reconstructed image by inverse affine transformed image V is the vertical coordinate in cylinder space for the image reconstructed by inverse affine transformation, and M represents the affine transformation matrix.

The reverse projection unit 58 reconstructs the three-dimensional image existing in the original three-dimensional space through reverse projection of the image existing in the two-dimensional cylinder space. The 3D image thus restored has 3D dense feature point coordinates and color information corresponding to the 2D dense feature point coordinates. Information about the 3D image may be obtained by using a rotation angle and a radius of the image information existing in the cylinder space according to Equation 2 below.

[Equation 2]

Here, θ is a rotation angle of the cylinder coordinate system. Referring to FIG. 3, the rotation angle may be defined as having a negative value on the left side and a positive value on the left side, for example. T is the angle corresponding to the horizontal unit length of the cylinder coordinate system, b is the offset of the rotation angle obtained from the cylinder coordinate, R is the rotation radius obtained from the cylinder coordinate, and r ( u , v ) is the ( u , v ) coordinate. Corresponding radius value 702, x, y, z are three-dimensional image coordinates, L is the length corresponding to the longitudinal unit length of the cylinder coordinate system, b is the offset of the z coordinate obtained from the cylinder coordinates.

The model generator 60 generates a 3D image model using the reconstructed 3D images. The model generator of the present embodiment generates a three-dimensional model through PCA (Principle Component Analysis), and includes a principal component analyzer 62, a three-dimensional shape model generator 64, and a three-dimensional texture model generator 66. It is provided, including. The principal component analyzer 62 performs principal component analysis on the reconstructed 3D image information. For example, the principal component analyzer 62 may perform principal component analysis on the coordinates and color information of the 3D facial feature point. The three-dimensional shape model generator 64 generates a three-dimensional shape model including an average and a variance vector of three-dimensional coordinates according to the principal component analysis result. In addition, the 3D texture model generator 66 generates a 3D texture model including an average and a variance vector of the color information of the 3D coordinates according to the principal component analysis result.

In the present embodiment, the present invention has been described with PCA as an example, but it is also possible to generate a 3D image model using techniques such as FDA (Independent Component Analysis), ICA (Independent Component Analysis), and LDA (Linear Discriminant Analysis). . In particular, in the case of the PCA method, since the image is represented with the minimum dimension, the calculation amount is minimized, and an optimized basis vector can be generated by analyzing the most important common characteristics in the face image.

The 3D image model generated by the model generator may represent a shape basis and a texture basis capable of expressing a 3D image as a linear model shown in Equation 3 below.

[Equation 3]

Where S is the shape of an arbitrary three-dimensional image, S _o is the mean (average of the shape) of the three-dimensional image coordinates, S _i is the i-th dispersion vector of the three-dimensional image coordinates, and α _i is the respective dispersion vector. Is a coefficient of color information (texture) of an arbitrary three-dimensional image, T ₀ is an average of color information according to three-dimensional image coordinates, and T _i is a j-th dispersion vector of color information of the three-dimensional image coordinates. Β _i is a coefficient for each dispersion vector, n is the number of variance vectors of the 3D image coordinates, and m is the number of variance vectors of the color information of the 3D image coordinates. If the coefficients α _i and β _i are adjusted according to Equation 3, an arbitrary three-dimensional image corresponding to the two-dimensional face image may be generated by the three-dimensional image model.

5 is a flowchart illustrating a method of generating a 3D image model according to an embodiment of the present invention. The 3D image model generating method illustrated in FIG. 5 includes the following steps performed in time series in the 3D image model generating apparatus 1.

In operation 210, the 3D image feature point selector 20 selects image feature points according to each of the 3D scan images, and generates triangular meshes according to the selected image feature points. As an example of a method of selecting image feature points, in the case of a face image, feature points such as eye tail, eyebrows, nostrils, and lips, which are easily distinguishable from other regions, may be selected based on a template prepared for images around the feature points. There is an image matching method for finding a position most similar to a template. In addition, a method for detecting feature points of a training face image using a detector trained with AdaBoost on the image around each feature point, a method of calculating the probability of each pixel being a feature point, and finding a probability mean position by learning a probability model, etc. There is this.

In operation 220, the two-dimensional space projector 30 projects the two-dimensional cylinder space. In particular, the two-dimensional space projecting unit 30 preferably projects the three-dimensional scanned image in two-dimensional space having two axes of rotation angle and height rotating about a predetermined central axis.

In operation 230, the image transform unit 40 transforms the projected images into a reference image. In detail, the step may include determining one image from the projected two-dimensional images as a reference image; Selecting feature points located at an outer side from among the determined feature points of the reference image, and determining a mask area bordering a line connecting the selected feature points; And performing affine transformation on the basis of the selected feature points or mask area to transform images other than the reference image to correspond to the reference image.

In operation 240, the 3D image reconstructor 50 determines a dense feature point and a mesh of the 2D image from the image transformed in operation 230. In this step, the 3D image reconstructor 50 extracts each coordinate or pixel included in the mask area as a feature point of the 2D image, and generates triangular meshes connecting the extracted dense feature points.

In operation 250, the 3D image reconstructor 50 reconstructs the 3D image through reverse fishnet transformation and reverse projection. The 3D image reconstructor 50 converts the images in which the triangular mesh is generated into an image existing in the 2D cylinder space through inverse affine transformation, and performs original projection through the reverse projection of the image existing in the 2D cylinder space. Restores the 3D image existing in the 3D space.

In operation 260, the 3D image model generator 60 generates a 3D image model using the reconstructed 3D images. In the case of a face image, the 3D image model generator 60 may select a basis vector effective for face recognition through principal component analysis of coordinates and color information of a 3D face feature point. The three-dimensional image model generator 60 is a three-dimensional shape model consisting of the mean and the variance vector of the three-dimensional coordinates according to the result of the principal component analysis, and a three-dimensional texture model consisting of the mean and variance vector of the color information of the three-dimensional coordinates Can be generated. An arbitrary three-dimensional face image may be generated by adjusting the coefficient of the generated three-dimensional coordinate dispersion vector and the coefficient of the color information dispersion vector.

Compared to the optical flow method of generating a three-dimensional face model using face image matching, it is difficult to distinguish adjacent pixels between the image of the ball and the forehead where the difference is very small in the face image. As a result, the light flow may be inaccurate. In addition, an iterative block image matching process is required to obtain a light flow result. In this case, the larger the block is, the larger the calculation amount is. However, when a predetermined number of feature points are used as in the present embodiment, a predetermined number of feature points are used. This operation has a small amount of computation because it performs one affine transformation for each triangular mesh. In addition, since the pixel value inside the triangle mesh is calculated by linear interpolation method, even in areas with uniform pixel values such as balls or foreheads, it is possible to accurately calculate the color and radius of the corresponding point and the corresponding point.

6 is a block diagram illustrating an image recognition device according to an embodiment of the present invention. The image recognition apparatus 300 illustrated in FIG. 6 includes a 3D scan image input unit 302, a 3D image feature point selector 304, a 2D space projector 306, an image deformer 308, and a 3D image. Restoring unit 310, model generating unit 312, 2D training image input unit 314, 3D image generating unit 316, first feature vector extractor 318, image input unit 320 and image determination unit 322. Among the above components, since the 3D scan image input unit 302 to the model generating unit 312 correspond to the elements shown in FIG. 1, common descriptions thereof will be omitted.

First, the image recognition apparatus 300 may further include an identification information acquisition unit (not shown) for main elements. For example, the identification information acquisition unit may be a feature point corresponding to the main elements (eyes, eyebrows, nose, mouth, etc.) of the face among the coordinates of the image existing in the two-dimensional cylinder space generated by the two-dimensional space projector 306. The sequence number of the three-dimensional face model in the boundary region connecting the outermost feature points for each major element may be stored as identification information of the corresponding major element.

The 2D training image input unit 314 receives 2D training images for use in prior learning for image recognition. For example, in order to build a face recognition system of a group, a 2D face image or training image of all members of the group is required. In order to construct a face recognition system, it is necessary to extract feature vectors through prior learning of training images.

The 3D image generator 316 generates a 3D composite image predicted from the 2D training image using the 3D image model.

The first feature vector extractor 318 projects the 3D composite image onto the 2D cylinder region, and extracts the feature vector according to each of the projected 2D images. The first feature vector extractor 318 may include an image controller (not shown) for adjusting various poses and illuminations for each training image, a projection unit (not shown) for projecting the image into a two-dimensional cylinder space, and a projected image. It includes a vector generator (not shown) for generating a learning vector for image recognition from the.

7 is a reference diagram illustrating an example of applying various rotations and movements to a two-dimensional training face image and extracting main elements. In the case of a face image, the pose and lighting controller obtains various three-dimensional face images by applying various poses and lighting conditions to the three-dimensional face image generated by the three-dimensional image generator 316. . The projection unit projects the acquired images on the two-dimensional cylinder space. The vector generator extracts a sub image according to main elements (eye + eyebrow, nose, mouth) of the 2D face image as a training image, and extracts a feature vector from the extracted training image. However, since the sizes of the sub-images for the main elements of the face are different from each other, it is necessary to enlarge or reduce the size of the sub-images and normalize them before generating the learning vector.

In this embodiment, the vector generator (not shown) extracts and extracts a two-dimensional sub-image by obtaining the maximum coordinates and the minimum coordinates of facial feature points corresponding to each facial element using identification information on the main elements of the face. It is preferable to generate a learning vector for face recognition as a first feature vector using the sub-image. The first feature vector includes, for example, a vector obtained by rearranging the intensity and the edge components of the intensity as one-dimensional vectors according to the main element image of the face. The generated feature vector may be used as an input vector for learning of the neural network learner 326 to be described later.

The image input unit 320 receives a 2D image of the input image to be recognized. The image determiner 322 includes a second feature vector extractor 324 and a neural network learner 326.

The second feature vector extractor 324 extracts a second feature vector from the input image. The second feature vector extractor 324 further includes a face region extractor (not shown), a key element detector (not shown), and a learning vector generator (not shown). In the input image, a face region and a background region to be detected are mixed, and the face region extractor detects a face region using a face color or an AdaBoost method. The key element detector detects an approximate position of the key elements through intensity histograms in the detected face region, and uses a probabilistic model (eg, a Gaussian model) trained with the image of the detected key elements. Detect. The learning vector generator generates a learning vector in which intensity and edge components are rearranged into one-dimensional vectors according to the detected main elements. In this document, the generated learning vector is called a second feature vector.

The neural network learner 326 receives the second feature vector and calculates a recognition result of the input image by calculating a similarity between the second feature vector and the first feature vector for the already extracted and stored training image. When a similar first feature vector is found as a result of the neural network learner 326, the ID of the person according to the found first feature vector is output. If no similar first feature vector is found, then output a message as a result of authentication denial.

8 is a flowchart illustrating an image recognition method according to an embodiment of the present invention. The image recognition method illustrated in FIG. 8 includes the following steps performed in time series by the image recognition apparatus 300.

In operation 410, the 3D image generator 316 generates a 3D image predicted from the 2D training image.

FIG. 9 is a detailed flowchart of step 410 of FIG. 8. The 3D image generator 316 receives the 2D training image (step 412), initializes the 3D image model (step 414), and the 3D image generator 316 receives the 2D training image and the 3D image. In order to minimize the difference between the 3D image reconstructed into the image model, the parameters of the 3D image model, that is, the coefficient of the 3D coordinate dispersion vector and the coefficient of the color information dispersion vector are determined (step 416), and the 3D image according to the determined coefficient is determined. Create (step 418). In this case, the initialization of the image model means that the 3D image model is parallelly moved or rotated so that the 3D image model approximately matches the size and position of the 2D image.

In operation 420, the first feature vector extractor 318 projects the 3D images generated in operation 410 into a 2D space. Prior to this step, a plurality of different 3D images may be generated by applying various poses and lighting to each of the 3D images, and when the first feature vector is extracted from the 3D images, the feature vectors robust to the change of the lighting of the pose may be extracted. have.

In operation 430, the first feature vector extractor 318 extracts a sub image using identification information about a main element. In operation 440, the first feature vector extractor 318 extracts a feature vector from each of the sub-images extracted in operation 430. In operation 450, the second feature vector extractor 324 extracts a sub image from an input image to be recognized.

In operation 460, the second feature vector extractor 324 extracts a second feature vector corresponding to the first feature vector from each of the extracted sub-images. Prior to extracting the second feature vector, the method may further include detecting a face region and main elements of the face from the input image. The facial region may be detected through conventional face detection methods such as face color or Adaboost method. The main elements of the face are detected using intensity histograms and the key elements of the training face images in the detected face area using a previously learned probability model. The second feature vector of this step is preferably a vector in which the intensity and edge components of the detected main elements are rearranged into one-dimensional vectors.

Conventionally, when a 2D image to be recognized is reconstructed into a 3D image, processing time is long. However, in the present embodiment, since the 3D image reconstruction process is not performed, the recognition speed of the image may be improved. In the present invention, since the two-dimensional image obtained by variously changing the pose and lighting of the face is used as training data in the process of calculating the feature vector from the training image information, the robustness of face recognition can be maintained.

In operation 470, the neural network learner 326 calculates similarity between feature vectors. There is no particular limitation on the method of calculating the similarity between feature vectors, but similar feature vectors may be searched using, for example, cosine distance, Euclidean distance, Maharaobis distance, and the like.

In operation 480, the neural network learner 326 calculates a recognition result. If the cosine distance between the feature vectors according to the two face images contrasted in step 470 is smaller than the predetermined reference value, the cosine distance is determined to be the same face.

Meanwhile, the 3D image model generation method and the image recognition method of the present invention can be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which may be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will understand that the present invention can be embodied in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown not in the above description but in the claims, and all differences within the scope should be construed as being included in the present invention.

Since the method and apparatus for generating a 3D image model of the present invention can improve the processing speed and accuracy required to generate a 3D image model, it is suitable for use in a 3D image synthesis apparatus and a 3D image recognition apparatus. In addition, the image recognition apparatus of the present invention can recognize a strong image to pose and lighting without restoring the two-dimensional input image to be recognized as a three-dimensional image, it is necessary to restore the two-dimensional input image to a three-dimensional image This reduces the processing time required, which improves the efficiency of the system.

1 is a block diagram illustrating an apparatus for generating a 3D image model, according to an exemplary embodiment.

2A and 2B are reference diagrams illustrating an example of extracting feature points from a 3D face scan image and generating a triangle mesh.

3 is a reference diagram illustrating an example of projecting a 3D face scan image to a 2D cylinder space.

Figure 4 shows an example of affine transformation of the image in the two-dimensional cylinder space in the present invention.

5 is a flowchart illustrating a method of generating a 3D image model according to an embodiment of the present invention.

6 is a block diagram illustrating an image recognition device according to an embodiment of the present invention.

7 is a reference diagram illustrating an example of applying various rotations and movements to a two-dimensional training face image and extracting main elements.

8 is a flowchart illustrating an image recognition method according to an embodiment of the present invention.

FIG. 9 is a detailed flowchart of step 410 of FIG. 8.

Claims (21)

a) selecting three-dimensional image feature points according to each of the three-dimensional scanned image; b) projecting each of the three-dimensional scanned images in a two-dimensional space; c) determining one image from the projected two-dimensional images as a reference image, and deforming the two-dimensional images other than the reference image to correspond to the shape of the reference image; d) extracting two-dimensional image feature points according to each of the deformed images, and reconstructing three-dimensional images using the extracted two-dimensional image feature points; And e) generating a 3D image model using the reconstructed 3D images. The method of claim 1, wherein step c) c1) determining one image from the projected two-dimensional images as a reference image; And c2) performing affine transformation so that the position of the feature points of the remaining images except for the reference image and the position of the feature points of the reference image correspond to each other. The method of claim 2, wherein c) is The method may further include determining a mask area according to feature points located at the outer side among the feature points of the reference image determined in step c1), and the affine transformation of step c2) is performed within the mask area. 3D image model generation method. The method of claim 3, wherein step d) d1) extracting points according to respective coordinates included in the mask area as two-dimensional image feature points; d2) generating triangular meshes according to the extracted two-dimensional image feature points; d3) generating images projected on the two-dimensional cylinder space by performing inverse affine transformation on the triangle meshes generated in step d2); And d4) reconstructing three-dimensional images by performing reverse projection on each of the images projected on the two-dimensional cylinder space generated in step d3). The method of claim 1, wherein b) b1) generating triangular meshes according to the selected feature points; And b2) projecting each of the three-dimensional scan images generated by the triangle meshes in a two-dimensional cylinder space. The method of claim 1, And generating a 3D image model by using principal component analysis (PCA) on the reconstructed 3D images. The method of claim 1, The 3D image model includes a 3D shape model and a 3D texture model, The three-dimensional shape model is a model based on a three-dimensional coordinate average and a variance vector generated through principal component analysis of coordinate information of three-dimensional feature points according to each of the restored three-dimensional images. The 3D texture model is a 3D image model generation, characterized in that the model according to the 3D color information mean and the variance vector generated through the principal component analysis of the color information of the 3D feature point according to each of the reconstructed 3D image Way. The method of claim 1, wherein step e) e1) obtaining 3D coordinate information and color information of feature points according to each of the reconstructed 3D images; And e2) generating a 3D image model for synthesizing an arbitrary 3D image using the acquired position information and color information. A computer-readable recording medium having recorded thereon a program for executing the method of generating a 3D image model according to any one of claims 1 to 8. A 3D image feature point selector for selecting 3D image feature points according to each of the 3D scan images; A two-dimensional space projector configured to project each of the three-dimensional scanned images into a two-dimensional space; An image transformation unit configured to determine one image from the projected two-dimensional images as a reference image, and to modify the remaining two-dimensional images other than the reference image to correspond to the shape of the reference image; A 3D image reconstructing unit which extracts 2D image feature points according to each of the modified images, and reconstructs 3D images using the extracted 2D image feature points; And And a model generator for generating a 3D image model using the reconstructed 3D images. The method of claim 10, wherein the image transformation unit, A reference image determiner which determines one image from the projected two-dimensional images as a reference image; A mask area determiner configured to determine a mask area according to feature points located at the outside of the feature points of the reference image; And And an affine transform unit that performs an affine transformation so that the position of the feature points of the remaining images except for the reference image and the position of the feature points of the reference image correspond to each other. The method of claim 10, The 3D image feature point selector further includes a mesh generator configured to generate triangular meshes according to the selected feature points. The two-dimensional space projection unit comprises a projection unit for projecting each of the three-dimensional scan image generated by the triangular mesh in the two-dimensional cylinder space. a) selecting three-dimensional image feature points according to each of the three-dimensional scanned images, and projecting each of the three-dimensional scanned images in a two-dimensional space; b) determining one image from the projected two-dimensional images as a reference image, and deforming the other two-dimensional images except for the reference image to correspond to the shape of the reference image; c) extracting two-dimensional image feature points according to each of the deformed images, and reconstructing a three-dimensional image by using the extracted two-dimensional image feature points, respectively; d) generating a 3D image model using the reconstructed 3D images; e) receiving 2D training images and generating 3D composite images predicted from the training images using the 3D image model; f) extracting a feature vector according to each training image from the generated three-dimensional composite images; And g) receiving a 2D image of the input image to be recognized, extracting a feature vector from the input 2D image, and using the similarity between the feature vector according to the input image and the feature vectors according to the training image; Calculating a recognition result for the image. The method of claim 13, The method may further include obtaining identification information about a main element constituting the image using the 2D images projected in step a). Step f) f1) The three-dimensional composite images generated in the step e) are projected in the two-dimensional space by adjusting the pose or the illumination, respectively, and the main images from the images projected in the two-dimensional space using the identification information of the main elements. Extracting sub-images related to the element; And f2) normalizing the extracted sub-images, and extracting feature vectors from the normalized sub-images. The method of claim 14, wherein obtaining the identification information comprises: Each of the projected two-dimensional images is classified according to a main element, and a feature point order value defined by pixels within a boundary region of feature points located at an outer side among feature points included in the divided main elements Image recognition method characterized in that obtained by the identification information. The method of claim 14, The feature vector includes an intensity component and an edge component of the normalized sub-images. The method of claim 14, Generating modified 3D images by applying a predetermined rotational movement and positional movement to the 3D composite images generated in step e), In step f1), the modified 3D images are respectively projected in a 2D space, and sub images related to the main element are extracted from the images projected in the 2D space by using identification information of the main elements. Further comprising: The feature vector extracted in step f2) is a learning vector for neural network learning. Calculating a recognition result for the input image using the similarity between the feature vector according to the input image and the feature vector according to the training image in step g) is performed on the training vector for the neural network learning and the input image. And image similarity between the feature vectors. The method of claim 13, wherein step e) e1) receiving two-dimensional training images; e2) initializing the 3D image model for each of the training images; e3) determining each variable of the 3D image model that matches the training image; And e4) generating a three-dimensional synthesized image predicted according to the determined variable, respectively. A computer-readable recording medium having recorded thereon a program for performing the image recognition method of claim 13 on a computer. A 3D image feature point selector for selecting 3D image feature points according to each of the 3D scan images; A two-dimensional space projector configured to project each of the three-dimensional scanned images into a two-dimensional space; An image transformation unit which determines one image from the projected two-dimensional images as a reference image, and deforms the remaining two-dimensional images other than the reference image to correspond to the shape of the reference image; A 3D image reconstructing unit which extracts 2D image feature points according to each of the modified images, and reconstructs a 3D image using the extracted 2D image feature points, respectively; A model generator for generating a 3D image model using the reconstructed 3D images; A 3D image generator which receives 2D training images and generates 3D composite images predicted from the training images using the 3D image model; A feature vector extracting unit extracting a feature vector according to each training image from the generated 3D composite images; An image input unit configured to receive a 2D image of the input image to be recognized; And And an image determining unit configured to extract a feature vector for the input image and calculate a recognition result for the input image by using similarity between the feature vector according to the input image and the feature vector according to the training image. Video recognition device. The method of claim 20, Further comprising an identification information acquisition unit for obtaining the identification information for the main elements constituting the image by using the projected two-dimensional images, The feature vector extractor may include an image controller configured to adjust various poses and lightings for each training image; A projection unit for projecting the image into a two-dimensional cylinder space; And And a vector generator for generating a feature vector for image recognition from the projected image.

Images