KR100889854B1 - Method and Apparatus for creating caricature Video - Google Patents

Abstract

The present invention discloses a method and apparatus for caricature video generation. The method of generating a caricature video of the present invention includes the steps of calculating a variation between feature points extracted from an input image and feature points of a 3D model image prepared in advance; Generating depth information of feature points extracted from the input image by using feature points of the 3D model image and the calculated variation; And transforming feature points of the input image having the generated depth information into exaggerated feature points according to a predetermined exaggeration algorithm, and generating a caricature image of the input image based on the exaggerated feature points. According to the present invention, it is possible to effectively extract the features of the input image, to generate a caricature video reflecting the characteristics of the input image, and to generate a cartoon effect by inputting a video including a face-oriented face. Facial images can be obtained. In addition, when the caricature video generation method of the present invention is applied to characters and avatars used in games, web, and mobile application services, realistic expressions are possible as compared to those based on templates.

Caricature, Exaggerated, Unrealistic Face Animation, 3D

Description

Method and Apparatus for Creating Caricature Video {Method and Apparatus for creating caricature Video}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for generating a caricature video, and more particularly, to a method and apparatus for generating a caricature video exaggerating a character image or an avatar image utilizing a user's personality in a game, animation, web site, and the like.

In general, the caricature is to satirize a person or anthropomorphic animals and plants. The caricature of a person's portrait image is an exaggeration of the characteristic elements that distinguish it from other people, and because it expresses a person's personality, it gives a more intense impression than a portrait drawn in reality.

Research on how to create caricatures more easily using computer graphics technology has been done. For example, Akleman, Brennan and Fujiwara create caricatures through the synthesis of faces. I've done it. In addition, Liang and Chiang created caricature images of new inputs based on works by caricature authors. However, the above methods are caricature generation methods considered for still images, and are not suitable for video caricatures showing continuous changes.

In addition, studies have been conducted to cartoonize live image and video. Wang and Holger applied cartoon-like effects to the input image or video using MeanShift splitting techniques or bilateral filters. However, these methods change the rendering style by applying abstraction to the input image or video, but they are not related to the method of expressing the unrealistic effects that appear in the actual cartoon.

As a related patent document, Korean Patent Laid-Open Publication No. 2000-47501 discloses a method of normalizing a face using a Gabor filter response, detecting a face feature through an ASM, and generating matching information with a Fourier descriptor.

Korean Patent Laid-Open Publication No. 2004-32452 extracts feature point information of facial components by using ASM technique specialized for faces, and transforms a three-dimensional polygon face basic model based on the three-dimensional caricature with high similarity to the user's face. It has been disclosed how to produce.

The present invention extracts the rotation information of the input image using the 3D model image, and generates a caricature video for the input image based on the extracted rotation information, thereby effectively generating a caricature image for the front image and the image close to the front It is an object of the present invention to provide a method and apparatus for generating a caricature video that can effectively reflect the characteristics of an input image and also obtain a non-realistic image that is cartoonly exaggerated.

According to an aspect of the present invention, there is provided a method for generating a caricature video, comprising: calculating a variation between feature points extracted from an input image and feature points of a 3D model image prepared in advance; Generating depth information of feature points extracted from the input image by using feature points of the 3D model image and the calculated variation; And transforming feature points of the input image having the generated depth information into exaggerated feature points according to a predetermined exaggeration algorithm, and generating a caricature video of the input image based on the exaggerated feature points. In the present invention, the variation refers to rotation information and position movement information of the 3D model image or the input image which are necessary to minimize the difference between the feature points extracted from the input image and the feature points of the 3D model image. Here, using the feature points of the 3D model image and the calculated variation, for example, depth information of the feature points of the 3D model image in which the feature points of the 3D model image are rotated and / or moved in accordance with the rotation information and the position movement information. Means as depth information of feature points of the input image.

According to another aspect of the present invention, there is provided a caricature video generating apparatus comprising: a disparity calculator configured to calculate a disparity between feature points extracted from an input image and feature points of a 3D model image prepared in advance; A depth information generator configured to generate depth information of feature points extracted from the input image by using feature points and variations of the 3D model image; An exaggeration unit for transforming the feature points of the input image having the generated depth information into exaggerated feature points according to a predetermined exaggeration algorithm; And an image generator configured to generate a caricature image of the input image based on the exaggerated feature point.

In order to solve the above other technical problem, the present invention provides a computer-readable recording medium on which a program for performing the caricature video generating method on a computer is recorded.

According to the present invention, by extracting the rotation information of the input image using the 3D model image, and generates a caricature video for the input image based on the extracted rotation information, the feature of the input image is effectively extracted, the feature of the input image This well reflected caricature video can be generated. According to the present invention, a non-realistic face image having a cartoon effect can be obtained by inputting a video including a face-oriented face. When the caricature video generation method of the present invention is applied to characters and avatars used in games, web, and mobile application services, realistic expressions can be achieved rather than based on templates.

Hereinafter, the caricature video generating method and apparatus of the present invention will be described in detail with reference to the accompanying drawings and embodiments.

1 is a schematic diagram illustrating a caricature video generation algorithm of the present invention.

As shown in FIG. 1, the caricature video generation algorithm of the present invention can be roughly divided into a preprocessing step and an execution step. In the preprocessing phase, we build an Active Appearance Model (AAM), generate 3D model images, and define rules for exaggeration. In the execution step, the feature point of the input image, in particular, the input face image is extracted using AAM, and the rotation information of the input face is calculated using the 3D model image. In this case, the rotation information is deformation information that transforms the input face image into a front face image. Next, after rotating the input face image to the front based on the calculated rotation information, exaggerated feature points are obtained using a predetermined predetermined exaggeration rule, and the exaggerated feature points are rotated again in the original direction. Finally, after the input frame is warped with an exaggerated feature point, a caricature video may be generated by applying a catning effect.

In the preprocessing step, AAM is generated by learning the shape and texture information of the training image. While maintaining the shape of the object based on AAM, we can find the object by searching the part with the most similar texture for the new input based on the learned texture information. Labeled images are used as training data to represent shape and texture. In order to include more texture information and shape information, several images showing various shapes of objects are required. For example, 216 training images taken by eight people with different expressions in nine directions can be used to include as many different lighting conditions and poses as possible. Feature points are available.

2 is a block diagram illustrating an apparatus for generating a caricature video according to the present invention. The caricature video generating apparatus 1 of the present invention includes a feature point extractor 10, a 3D model image storage unit 20, a disparity calculation unit 30, a depth information generator 40, a front image converter 50, The image exaggeration unit 60 and the image generating unit 70 is included.

The feature point extractor 10 receives a video sequence and extracts feature points from each of the input video images using an AAM that is already constructed according to a predetermined algorithm.

The 3D model image storage unit 20 stores information about the 3D model image obtained through pre-training the training images. In particular, the 3D model image information includes information on feature points already extracted for the 3D model image.

3 is a reference diagram showing an example of a training image for generating a 3D model image in the present invention. In order to generate the 3D model image, the feature points extracted from the front training image of one of the training images are determined as the feature points of the initial 3D model, and the distance between the feature points of the initial 3D model and the feature points according to each of the training images is determined. After calculating the minimum positional and rotational shifts, it is necessary to update the feature points of the initial 3D model using the calculated positional and rotational shifts. Hereinafter, the principle of generating a 3D model image using the feature points of the plurality of training face images shown in FIG. 3 will be described in detail.

First, the matrix X of the initial 3D model image composed of n three-dimensional feature points may be expressed as in Equation 1 below. The initial 3D model image is one front image selected from the training images and has a z value of zero.

Equation 1

A matrix (x) for a training image consisting of n two-dimensional feature points is expressed by Equation 2 below.

Equation 2

In this embodiment, in order to calculate the difference between the feature point according to each training image and the initial 3D model image, the feature point of the initial 3D model image is projected and used in the same 2D plane as the training image.

Equation 3

여기에서, X는 3D 모델 영상 정보이고, R(c)는 X를 회전시키는 상기 복수의 회전 정보들을 포함하는 회전 매트릭스로, R(c)=Rx(rx)Ry(ry)Rz(rz)를 포함하고, Rx(rx)는 x축을 중심으로 회전되는 각도에 대한 회전정보, Ry(ry)는 y축을 중심으로 회전되는 각도에 대한 회전정보, Rz(rz)는 z축을 중심으로 회전되는 각도에 대한 회전정보이며, t(c)는 회전된 3D 모델 영상의 위치 이동 정보로 t(c)=(tx, ty, tz)를 포함하고, tx, ty, tz는 상기 복수의 회전 정보에 따른 각각의 축에서의 위치 이동 정보이며, x는 입력 영상 정보이다.

Here, X is 3D model image information, R (c) is a rotation matrix including the plurality of rotation information for rotating X, R (c) = Rx (rx) Ry (ry) Rz (rz), where Rx (rx) is centered on the x-axis Rotation information about the angle rotated by, Ry (ry) is rotation information about the angle rotated around the y-axis, Rz (rz) is rotation information about the angle rotated around the z-axis, t (c) is rotated T (c) = (tx, ty, tz) as position movement information of the 3D model image, wherein tx, ty, and tz are position movement information in respective axes according to the plurality of rotation information, and x is Input image information.

Here, P is a matrix for projecting a feature point of the 3D model onto the 2D image, and the rotation matrix R (c) may be expressed as follows.

Equation 4

, R(c) = R_x(r_x)R_y(r_y)R_z(r_z), t(c) = (t_x,t_y,t_z)

, R (c) = R _x (r _x ) R _y (r _y ) R _z (r _z ), t (c) = (t _x , t _y , t _z )

F (c) of Equation 3 is a problem of calculating how much the 3D initial 3D model image is rotated (R (c)) and moved (t (c)) to become similar to the current training image. In the present embodiment, the initial 3D model image is a face image facing to the front, and it can be seen how much it needs to rotate and move to face the training image through Equation 3 above. What remains is to solve the optimization problem of finding c = (r _x , r _y , r _z , t _x , t _y , t _z ) that minimizes f (c). In particular, in this embodiment, the optimization problem is preferably solved using Sequence Quadratic Programming (SQP). In this case, you can assign 0 to all elements as the initial value of c.

Next, the principle of updating the initial 3D model using the R (c) and t (c) calculated above will be described. The j th feature point X _j of the initial 3D model to be updated may be expressed as in Equation 5 below. In addition, an i-th training image consisting of a set of n feature points is referred to as x ⁱ , and may be expressed by Equation 6 below.

Equation 5

Equation 6

g (X _j ) is the difference between the distance between the j th feature point of the 3D model X projected by P and the j th feature point of the training image after the rotation movement / position movement process according to the previously estimated rotation / position movement information C _i . Is obtained. According to Equation 6, the difference between the distances between the j th feature points of all the training images may be summed and X _j may be obtained to minimize the summed values. A matrix P for projecting a 3D image onto a 2D image and t (c ⁱ ) representing position movement information are expressed by the following equation.

Equation 7

, R(cⁱ)= R_x(rⁱ _x)R_y(rⁱ _y)R_z(rⁱ _z), t(cⁱ) = (tⁱ _x,tⁱ _y,tⁱ _z)

, R (c ⁱ ) = R _x (r ⁱ _x ) R _y (r ⁱ _y ) R _z (r ⁱ _z ), t (c ⁱ ) = (t ⁱ _x , t ⁱ _y , t ⁱ _z )

Using Equation 7, the current 3D model image can be rotated using the previously obtained c, and then projected onto a 2D image to be compared with each training image. In addition, by minimizing g (X _j ) to obtain the feature point X _{j such} that the rotated feature points of the 3D model are as close as possible to the feature points of the training image, the 3D model image may be updated. That is, the updated new 3D model image is obtained from more training images, and more accurately reflects the characteristics of the 3D object. The optimization problem in Equation 7 can be solved quickly using the psuedo-inverse technique. 4 is a reference diagram illustrating an example of a 3D model image used for the caricature video generation of the present invention.

Since the 3D model image generated according to the method is generated based on the training images, it can be said to represent the shape of the training image. The 3D model image constructed in this way is composed of only a few simple feature points compared to the forms generated by the studies on the existing 3D reconstruction. However, in the case of the present invention, since the face caricature is generated based on the feature points, complicated mesh information or a specific face shape is not required.

The variation calculator 30 calculates a variation between the feature points extracted from the input image and the feature points extracted from the 3D model image. The variation calculator 30 includes an image matcher 32 and a variation selector 34. The image matching unit 32 deforms the feature points of the 3D model image according to a plurality of predetermined rotation information and position movement information, and matches the modified feature points with the feature points of the input image. The variation selector 34 has the smallest or most similarity between the feature points of the input image and the feature points obtained by rotating / positioning the feature points of the 3D model image among the plurality of rotation information and the position movement information. The rotation information and the position shift information are selected as the rotation shift and the position shift shift. In particular, the shift selector 34 projects the 3D model image modified according to the rotation information and the position movement information on the same two-dimensional plane as the input image, and then compares the projected feature points with the feature points of the input image. Do.

The rotation shift and the position shift shift between the feature points of the input image and the feature points of the 3D model image may be calculated through the above Equations 1 to 4. In this case, x is a matrix representing feature points of the input image, and X is a matrix representing feature points of the 3D model image. In Equation 3, R (c) and t (c), which minimize f (c), become rotational shifts and positional shifts, respectively, according to the input image.

The depth information generator 40 generates depth information according to the feature points of the input image by using the feature points of the 3D model image and the calculated rotation shift and position shift shift. In particular, the depth information generator 40 rotates the 3D model image according to the calculated rotation shift, and then moves the depth information among the 3D information of the 3D model image obtained by positioning the 3D model image again according to the position shift variation. Treat it as information.

The front image converting unit 50 generates the front image by correcting 3D input images having depth information generated by the depth information generating unit 40 according to the calculated rotation shift and position shift shift.

The image exaggeration unit 60 exaggerates the feature points of the front image generated by the front image conversion unit according to a predetermined front image exaggeration algorithm. In the present embodiment, the exaggeration rule for the caricature image is largely divided into an inbitwinner exaggeration for exaggerating the ratio of the inside of the face and a component exaggeration for exaggerating the shape of the face. Each exaggeration applies an exaggeration with different formulas, but basically it compares input image and 3D model image and emphasizes the difference.

For example, in the case of face image exaggeration, the previously defined Inbit Winner can be used to exaggerate the ratio inside the face. InBitWinner is a proportional model defined for faces.

FIG. 5 is a conceptual diagram illustrating an example of an inbit winner for face image exaggeration. In order to construct Inbit Winner, horizontal and vertical lines may be set at corresponding facial feature points, and the proportional information of the face may be extracted based on this. In FIG. 5, (a) an image is an inbit winner of an average face, and (b) an image is an inbit winner of an input face. Next, the difference between the two faces may be calculated by comparing the intervals between the lines of the two inbit winners, and the interval may be adjusted to generate an exaggerated face. In FIG. 5, the image (c) shows an example in which the distance between the nose parts is wider when the weight of the nose in the interval of the given face is higher than the weight of the nose in the interval of the average face. In FIG. 5, the (d) image is an example of applying an exaggeration method similar to the aforementioned method to exaggerate the ratio of the entire face. You can't exaggerate the individual shapes of facial elements through InbitWinner Exaggeration. Therefore, in order to exaggerate the shape of the facial elements, it is desirable to apply the exaggeration to the face shape, eyes, nose and mouth separately.

6 is a reference diagram for explaining a concept of individual form exaggeration for face image exaggeration. In FIG. 6, the face is divided into four areas based on eyes and nose to exaggerate the shape of the face, and four squares are represented by yellow lines. The slenderness and roundness of the face can be obtained using the distance between the line segment connecting the point p1 and the point p2 (image of FIG. 6 (b)) and the average point of the points belonging to each rectangle. The shape of the rectangle may be adjusted to exaggerate the shape of the jaw and the face ((c) image of FIG. 6).

For example, in order to exaggerate the size of the eyes, nose, and mouth, the average face and the input face measure the relative sizes of the components, and then an exaggeration rule as shown in Equation 8 may be applied.

Equation 8

Here, x _i represents the i th feature point corresponding to one component, and x _mean is the average value of the feature points corresponding to one component. r is a factor representing the size difference between the input face and the average face, and the constant s is a scaling parameter, which is 0.7 in this example.

7 is an image illustrating a result of exaggerating the size of a component according to an embodiment of the present invention. 7 (a) shows the components of the average face, (b) shows the components of the input face, and (c) shows the result of exaggeration through the exaggeration of size.

In order to exaggerate each shape of the facial component, a linear transformation according to Equation 9 may be used.

Equation 9

8 is an image showing a result of exaggerating the shape of a component according to an embodiment of the present invention. The image (a) of FIG. 8 represents a component of an average face, (b) the image represents a component of an input face, and (c) the image represents a result image exaggerated through shape exaggeration. In FIG. 8, in order to exaggerate the eye, a rotation matrix T is used in which feature points corresponding to the eye are rotated by θ representing a difference between the angle of the eye at the input face and the angle of the eye at the average face. The angle of the eye can be measured by the slope of the line connecting the two ends of the eye. In FIG. 8, only the far end of the nose was moved to exaggerate the shape of the nose. The r value represents the distance between the tip of the nose and the mean of the feature points that make up the nose. s is the size constant, 0.9 was used in this example. Shear transformation through a was applied to exaggerate the shape of the lips. In this case, a may be obtained by a difference between the inclination of the line segment connecting the average point and the end point of the feature points constituting the mouth in the input face and the inclination obtained from the average face.

The image generator 70 generates a caricature image of the input image based on the exaggerated feature points. In particular, the image generating unit 70 includes a restoring unit 72 and a warping processing unit 74. The restoring unit 72 transforms the exaggerated feature points into reconstructed feature points according to the variation, and the warping processing unit 74 The warping of the input image is performed based on the restored feature points. In addition, the image generator 70 may further include a katuning processor (not shown). The katuning processor generates a cartoonized face image through a katuning process on the warped image.

9 illustrates a caricature image generated according to an embodiment of the present invention. In FIG. 9A, a red dot is a feature point extracted from an input face, and a blue dot is an exaggerated feature point by applying an exaggeration rule. The image (b) of FIG. 9 is a cartoon-style image obtained by warping an original feature point of an input face with an exaggerated feature point and performing a katuning process.

 10 illustrates a portion of a caricature video generated in accordance with one embodiment of the present invention. In the case of the example shown in the upper part of FIG. 10, a more severe exaggeration was applied by increasing the size control constant when determining the degree of exaggeration, and the example shown in the lower part is an example of automatically applying the exaggeration.

11 is a flowchart illustrating a caricature video generation method according to an embodiment of the present invention. The caricature video generating method of FIG. 11 includes the following steps performed in time series on the caricature video generating apparatus 1.

In operation 110, the feature point extractor 10 receives an input image and extracts feature points from the received input image.

In step 120, the variation calculator 30 calculates a rotation variation and a position variation. In particular, the image matching unit 32 transforms the feature points of the 3D model image stored in the 3D model image storage unit according to preset rotation information and / or position information, performs matching with the feature points of the input image, and performs a shift. The selector 34 selects rotation information and / or position information that minimizes the difference between the feature points of the 3D model image and the feature points of the input image.

In operation 130, the depth information generator 40 generates depth information of feature points of the input image. When the rotation information and the position information that minimize the difference between the 3D model image and the input image are respectively specified in step 120, the information about the feature points of the 3D model image modified according to the specified rotation information and the position information, respectively, is specified. . Since the input image information is two-dimensional information and does not include depth information, depth information according to feature points of the input image may be calculated by using depth information according to feature points of the modified 3D model image. In the present embodiment, the depth information of the feature points of the deformed 3D model image is treated as depth information according to the feature points of the input image.

In operation 140, the front image converting unit 50 converts the input image having depth information into a front image by rotating and / or moving the input image having depth information. In this case, the information about the rotational movement and the positional movement may preferably use the variation information selected through the variation selection unit 34.

In step 150, the image exaggeration unit 60 exaggerates the feature points of the front image according to the image exaggeration algorithm, in particular the front image exaggeration algorithm. An example of the image exaggeration algorithm is the inbitwinner exaggeration algorithm or the algorithm according to the exaggeration of the shape of the component.

In step 160, the restoration unit 72 transforms the exaggerated three-dimensional feature points into reconstructed feature points by rotating and / or repositioning the three-dimensional feature points.

In operation 170, the warping processor 74 generates an exaggerated image by performing warping on the input image based on the restored feature points. The method may further include applying a katuning effect to the warped image.

12 is a conceptual diagram of a caricature generation method according to an embodiment of the present invention. A method of generating a caricature based on an AAM, a 3D reference model, and an exaggeration rule pre-built in the preprocessing step is described with reference to FIG. 10. Image (a) of FIG. 12 is an example of facial feature points extracted from an input video using an object tracking technique of AAM. The rotation information c of the input face is calculated according to Equation 3, the 3D model is rotated by c to be similar to the direction of the input face, and the z values of the feature points of the 3D model image are substituted as the z values of the feature points of the input face. do. When the feature point of the input face is rotated by -c, the feature point according to the front face may be obtained (image of FIG. 12B). Next, the exaggeration rule based on the front face previously determined is applied to exaggerate the feature points of the input face facing the front face (figure (c) of FIG. 12) and rotate it again to face the original direction of rotation (FIG. 12). (D) video of 12). Finally, after performing warping based on the feature points according to the image of FIG. 12 (d), a caricature may be obtained by applying a tuning technique.

Meanwhile, the present invention can be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which may be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will understand that the present invention can be embodied in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown not in the above description but in the claims, and all differences within the scope should be construed as being included in the present invention.

The method and apparatus for generating a caricature video of the present invention can be usefully used for all contents related to the exaggeration of a character image or an avatar image utilizing a user's personality in the field of digital content, especially games, animations, and websites.

1 is a conceptual diagram illustrating a caricature video generation algorithm of the present invention.

2 is a block diagram illustrating an apparatus for generating a caricature video according to the present invention.

3 is a reference diagram showing an example of a training image for generating a 3D model image in the present invention.

4 is a reference diagram illustrating an example of a 3D model image used for the caricature video generation of the present invention.

5 is a reference diagram for explaining an example of an inbit winner for face image exaggeration.

6 is a reference diagram illustrating the concept of individual form exaggeration for face image exaggeration.

7 is an image illustrating a result of exaggerating the size of a component according to an embodiment of the present invention.

8 is an image showing a result of exaggerating the shape of a component according to an embodiment of the present invention.

9 illustrates a caricature image generated according to an embodiment of the present invention.

10 illustrates a portion of a caricature video generated according to an embodiment of the present invention.

11 is a flowchart illustrating a caricature video generating method according to an embodiment of the present invention.

12 is a conceptual diagram for a caricature video generating method of the present invention.

Claims (15)

a1) extracting feature points from an input image; a2) rotates the feature points of the 3D model image prepared according to the plurality of predetermined rotation information, parallelly moves the rotated image according to the plurality of predetermined position movement information, and then projects the positionally shifted image onto the two-dimensional plane Generating projected feature points; And a3) selecting rotation information that minimizes the difference between the projected feature points and the feature points of the input image as a rotation variation, and rotates R (c) and t (c) to minimize f (c) in the following equation. Selecting as a variation and a position variation; b) generating depth information of feature points extracted from the input image using the feature points of the 3D model image and the calculated variation; And c) transforming feature points of the input image having depth information generated through step b) into exaggerated feature points according to a predetermined exaggeration algorithm, and generating a caricature image of the input image based on the exaggerated feature points; Caricature video generation method comprising a. Equation

Here, X is 3D model image information, and R (c) is a rotation matrix including the plurality of rotation information for rotating X, where R (c) = Rx (rx) Ry (ry) Rz (rz). Rx (rx) is rotation information about an angle rotated about the x axis, Ry (ry) is rotation information about an angle rotated about the y axis, and Rz (rz) is an angle rotated about the z axis. T (c) is the positional movement information of the rotated 3D model image and includes t (c) = (tx, ty, tz), and tx, ty, and tz are respectively corresponding to the plurality of rotation information. Position movement information on an axis of x, and x is input image information. delete delete The method of claim 1, The step b) is a caricature video generation method, characterized in that to calculate the depth information of the feature points extracted from the input image using the depth information of the feature points according to the image obtained by rotating the 3D model image according to the variation. The method of claim 1, In step c), transforming the feature points having depth information into the exaggerated feature points according to a predetermined exaggeration algorithm in the step b), Caricature video generation, characterized in that for generating the feature points according to the front image by modifying the feature points having the depth information, and to transform the feature points according to the generated front image into exaggerated feature points through a predetermined exaggeration algorithm for the front image. Way. The method of claim 1, wherein step c) c1) transforming the feature points having depth information into feature points according to the front image according to the variation through step b); c2) transforming the feature points according to the front image into exaggerated feature points according to a predetermined front image exaggeration algorithm; c3) transforming the exaggerated feature points into feature points reconstructed according to the variation; And c4) generating a caricature image by warping the input image based on the restored feature points. delete delete The method of claim 6, The front image exaggeration algorithm is a caricature video generation method, characterized in that the algorithm according to the exaggerated exaggeration or exaggeration of the shape of the component. The method of claim 1, wherein the feature points of the 3D model image Determining feature points extracted from one front training image among the training images as feature points of the initial 3D model; Calculating position shift and rotation shift to minimize the distance between the feature points of the initial 3D model and the feature points according to each of the training images; And And updating the feature points of the initial 3D model by using the calculated positional and rotational variations. A computer-readable recording medium in which any one of claims 1, 4, 5, 6, 9, and 10 is recorded with a program for performing a method of producing a caricature video on a computer. A variation calculator for calculating a variation including rotation information and position movement information of the 3D model image or the input image required to minimize the difference between the feature points extracted from the input image and the feature points of the 3D model image; In the input image using the feature points and the variation of the 3D model image A depth information generator for generating depth information of extracted feature points; An image exaggeration unit for transforming feature points of the input image having the generated depth information into exaggerated feature points according to a predetermined exaggeration algorithm; And An image generator configured to generate a caricature image of an input image based on the exaggerated feature point, The variation calculator a1) extract the feature points from the input image, a2) rotates the feature points of the 3D model image prepared according to the plurality of predetermined rotation information, parallelly moves the rotated image according to the plurality of predetermined position movement information, and then projects the positionally shifted image onto the two-dimensional plane To generate the projected feature points, a3) calculates the variation by selecting rotation information and rotational position for minimizing the difference between the projected feature points and the feature points of the input image, wherein R (c) is minimized in the following equation: ) And t (c) are selected as a rotation shift and a position shift. Equation

Here, X is 3D model image information, and R (c) is a rotation matrix including the plurality of rotation information for rotating X, where R (c) = Rx (rx) Ry (ry) Rz (rz). Rx (rx) is rotation information about an angle rotated about the x axis, Ry (ry) is rotation information about an angle rotated about the y axis, and Rz (rz) is an angle rotated about the z axis. T (c) is the positional movement information of the rotated 3D model image and includes t (c) = (tx, ty, tz), and tx, ty, and tz are respectively corresponding to the plurality of rotation information. Position movement information on an axis of x, and x is input image information. delete The method of claim 12, The apparatus may further include a front image converter configured to transform feature points having depth information generated by the depth information generator into feature points according to the front image. The exaggeration unit is to transform the feature points according to the front image to the exaggerated feature points according to a predetermined front image exaggeration algorithm, The image generating unit may include a restoring unit configured to transform the exaggerated feature points into restoring feature points according to the variation, and further include a warping processing unit that performs warping on the input image based on the restored feature points. Caricature video generation device. The method of claim 12, wherein the variation calculator A matching unit to rotate the feature points of the 3D model image prepared according to the plurality of predetermined rotation information, project the rotated image onto a two-dimensional plane, and match the feature points of the input image; And And a variation selector for selecting rotation information that minimizes a difference between the feature points of the input image and the feature points of the projected 3D model image among the plurality of rotation information.

Images