KR101701199B1 - System and method for virtual cosmetic surgery by using mesh warping - Google Patents

Abstract

The present invention relates to a method and system for virtual cosmetic surgery, and more particularly, to a method and system for virtual cosmetic surgery that allows a user to freely shape a desired portion by deforming an image using mesh warping.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and system for virtual cosmetic surgery using mesh warping,

The present invention relates to a method and system for virtual cosmetic surgery, and more particularly, to a method and system for virtual cosmetic surgery that allows a user to freely shape a desired portion by deforming an image using mesh warping.

Today, cosmetic surgery has become an important place to overcome complexity, recovery of self-confidence, and furthermore, to influence interpersonal and sociability. There are various procedures such as incision insertion, Is not able to grasp the point of the desired operation of the patient and can not be formed. There is a problem that a person who can not grasp the possible degree of molding before consulting a plastic surgeon and can not grasp the desired facial face There is a problem that people are reluctant to visit plastic surgery.

In order to solve the above problems, a program such as an application for practicing a virtual cosmetic surgery in a form that a user has imagined using a terminal such as a smart phone has been developed.

Korean Patent No. 10-0764130 (prior art document 1) has a feature that a molding filter of a molding style desired by a user is manufactured in advance and molded.

However, the technique of the prior art document 1 can feel the heterogeneity due to the distortion of the image during the virtual molding, and can not compare the beauty of the output after the molding.

Prior Art 1: Korean Patent No. 10-0764130

In order to solve the above problems, the present invention is to provide a virtual molding method and system capable of automatically extracting contours of jaws, eyes, nose, and mouth from a facial image.

Another object of the present invention is to provide a tentative molding method and system for obtaining an output product that does not feel a sense of heterogeneity when an image is deformed using mesh warping.

It is another object of the present invention to provide a virtual molding method and system capable of comparing a molded image with a previously stored beautiful face in order to solve the above problems.

The present invention further provides a virtual molding method and system capable of stereoscopically representing a planar image in order to solve the above problems.

In order to attain the above object, the present invention provides a method for generating an image, comprising the steps of: generating outline coordinates located on an outline of an eye, nose, mouth, and jaw in a face photographed image; Creating a mesh group using the mesh point, shaping the image using mesh warping to the mesh group, and outputting the molded image. ≪ RTI ID = 0.0 > Provides a virtual cosmetic surgery method.

According to the present invention, since the outline of the eyes, nose, mouth and jaw is automatically extracted from the face image, the user can perform virtual molding easily and quickly.

Further, according to the present invention, it is possible to form more finely since it further includes fine adjustment of the coordinates and contour of the eyes, nose, mouth and jaw in the face image.

According to the present invention, there is further included a step of finely adjusting coordinates and contours of the eyes, nose, mouth, and jaw in the face image, so that coordinates and contours extracted from the unclear face image can be readjusted, There is a characteristic that molding can be performed.

Furthermore, according to the present invention, since mesh warping is performed with a set of meshes of three mesh points, a smoother image is output and finer shaping can be performed.

Furthermore, according to the present invention, the molding can be easily guided because it can be matched with the ratio of the beautiful person, or the output can be compared with the ratio of the beautiful person.

Further, according to the present invention, a 2D image can be converted into a 3D image and outputted in a solid form, so that more detailed modifications and results can be confirmed.

1 is a block diagram illustrating a virtual cosmetic surgery system using mesh warping according to an embodiment of the present invention.
2 is a block diagram illustrating a method of virtual cosmetic surgery using mesh warping according to an embodiment of the present invention.
3 is a block diagram illustrating a procedure for generating outline coordinates of a method of virtual cosmetic surgery using mesh warping according to an embodiment of the present invention.
FIGS. 4 to 15 are views showing steps of a virtual cosmetic surgery method using mesh warping according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The sizes and shapes of the components shown in the drawings may be exaggerated for clarity and convenience. In addition, terms defined in consideration of the configuration and operation of the present invention may be changed according to the intention or custom of the user, the operator. Definitions of these terms should be based on the content of this specification. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the inventions. Of course.

Referring to FIGS. 2 and 3, the method and system for virtual cosmetic surgery using mesh warping according to an embodiment of the present invention roughly includes the steps of extracting the contours of the eyes, nose, mouth, and jaw, (S100) of generating outline coordinates by using the outline coordinates, generating (110) a mesh point including outline coordinates on the face image by using the outline coordinates and arranging them at a predetermined interval in a lattice shape, (S120) generating a mesh group, a mesh warping step (130) of deforming the image by moving the mesh point and distorting it with the image, and outputting the deformed image.

In addition, the step S100 of generating the outline coordinates of the eyes, nose, mouth, and jaw in the face image may include extracting reference coordinates of the pupil using a face detector operator in the image (S200) (S210) of extracting reference coordinates of the nose, mouth, and jaw using the reference coordinates of the eye, nose, mouth, and jaw based on the reference coordinates of the extracted pupil, nose, (S230), and generating (S240) at least one outline coordinate located on the contour using the outlines of the generated eyes, nose, mouth, and jaw. At this time, steps (S220, S250) for finely adjusting the positions of the eyes, nose, mouth and jaw such that the positions of the eyes, nose, mouth and jaw of the actual face image coincide with the positions of the eyes, nose, . In detail, the reference coordinates of the pupil, nose, mouth, and jaw are finely adjusted after extracting reference coordinates of the eyes, nose, mouth, and jaw (S210) (S240) after extracting the outline coordinates of the eyes, nose, mouth, and jaw, and the outline coordinates of the eyes, nose, mouth, , The nose, the mouth, and the jaw (step S250).

On the other hand, creating a mesh group using a mesh point may include connecting three mesh points to create a triangular mesh, and creating a mesh group of triangular meshes.

The step of shaping the image using mesh warping to the mesh group then uses the mesh warping, in which the outline coordinates are shifted and the outline coordinates moved and the face image are moved together to distort the image.

In addition, the step of forming an image using mesh warping on the mesh group (S130) and the step of outputting the formed image may be performed by comparing the ratio of the eyes, nose, mouth, and jaw of the image to the golden ratio extracted from the face (Step S150) so as to perform effective virtual molding.

The step of outputting the mesh group to the mesh group using the mesh warping may include a stereoscopic output step (S170) of converting the 2D formed image into a 3D image and outputting the 3D image so as to be rotatable by a predetermined angle.

1, the terminal 10 of the present invention includes a processor 11, a storage unit 12, an output unit 13, and an input unit 14, and the processor 11 executes an operation Device, and the output unit 13 and the input unit 14 may be respectively provided, but they may be equipped with a touch screen like a smart terminal.

The processor 11 generates at least one or more contour coordinates located on the contour lines of the eyes, nose, mouth, and jaw in the face image, includes contour coordinates in the face image, generates mesh points arranged at predetermined intervals , Generates a mesh group using the mesh point, modifies the face image using mesh warping on the mesh point, and outputs the deformed image to the output unit.

For example, the processor 11 according to the present invention may be a processing device such as a CPU, an AP, and a GPU.

The output unit 13 displays the image output from the processor 11. [

The input unit 14 is means for receiving a fine adjustment value, and the processor 11 performs fine adjustment based on the input fine adjustment value.

The storage unit 12 stores information necessary for carrying out the respective steps. For example, the storage unit 12 may store the coordinates of the pupil, the mouth, and the jaw based on the reference coordinates of the extracted pupil. The pupil, the nose, the mouth, and the jaw of the pupil, the nose, the mouth, and the jaw, the reference pupil, the nose, the mouth, and the jaw, (S130) of shaping an image using mesh warping on the mesh group, and outputting the molded image. In the step of outputting the molded image, the eye, nose, mouth, and jaw of the face A unique moving weight corresponding to each direction of the mesh point, which is used in the stereoscopic output step (S150) for outputting the relative ratio, angle and coordinates and the formed image in three dimensions, is stored.

At this time, the information about the position ratio, the size ratio, the relative ratio of the face of the general person, the angle and the coordinates, and the moving weight are stored in the storage unit of the terminal in advance, The terminal may be provided with a communication unit 15 so as to collect information stored in the server through wired / wireless communication with an external server and proceed with each step. Of course, to be. Accordingly, the communication unit 15 stores the ratio, angle, and coordinates of the face image, the pupil, the nose, the mouth, and the jaw position ratio, the size ratio, and the eye, nose, 12). Hereinafter, each configuration and step will be described in detail.

A method of virtual cosmetic surgery using mesh warping according to an embodiment of the present invention includes a step of generating outline coordinates positioned on an outline of an eye, a nose, an mouth, and a jaw in an image in which a face is photographed. The detailed process of creating the outline coordinates located on the outline of the eyes, nose, mouth and jaw in the image is as follows.

First, a reference coordinate of a pupil is generated (S200) in an image of a face, a reference coordinate of a nose, a mouth, and a jaw is generated (S210) based on the reference coordinates of the generated pupil (S210) (S230) of creating an outline of the eyes, nose, mouth and jaw based on the reference coordinates of the nose, mouth, and jaw, and at least one outline coordinate S240 located on the outline of the eyes, nose, . Each step will be described in detail with reference to the drawings.

5, in step S100 of generating outline coordinates of eyes, nose, mouth, and jaw, a step S200 of creating reference coordinates of a pupil in a face image is firstly described. Specifically, In the face image, the coordinates of the pupil are extracted using a face detector operator, and reference coordinates of the eye are generated in the coordinates of the extracted pupil. Here, the reference coordinates are coordinates of a position The coordinates are input coordinates, and a detailed description will be described later. On the other hand, the Face Detector is an API (Application Programming Interface), and API is a language or a message used for communication between an operating system and an application program. In detail, Face Detector is a class embedded in API of Android platform provided by google, and has a function of recognizing and extracting a pupil from an image. An example of the source code of the Face Detector API is shown in Table 1 below.

Here private EyePose findEyeMidPoint (Bitmap face) is the operator that extracts the pupil, and FaceDetector.Face [] faces = new FaceDetector.Face [1]; Is a command to set the maximum number of recognizable faces. Also count = fd.findFaces (rgb565, faces); Is a command for executing the face recognition function, faces [i] .getMidPoint (midpoint); Eye.Distance = faces [i] .eyesDistance (); is a command to create a point in the middle of one pupil and another. Is a command to calculate the distance between one pupil and another.

Referring to FIG. 6, the reference coordinates of the pupil are extracted from the face image using the face detector, and reference coordinates of the nose, mouth, and jaw are generated based on the reference coordinates of the pupil In step S210, the average pupil, nose, mouth, and jaw position ratios of the ordinary people stored in the storage unit or stored in the server are retrieved and compared with the reference coordinates of the pupil. The reference coordinates of the nose, mouth, and jaw are generated at positions corresponding to the average pupil, nose, mouth, and jaw position ratios. Therefore, since the reference coordinates of the nose, mouth, and jaw are generated by comparing the average pupil, nose, mouth, and jaw position ratios of the general people stored in advance by the user, the positions of the nose, Reference coordinates of nose, mouth, and jaw may be generated at coordinates.

Referring to FIGS. 7 and 8, the outline of the eyes, nose, mouth, and jaw is then generated (S230) based on the reference coordinates of the pupil, nose, mouth, and jaw (S230) (Step S220) of fine-tuning the face image so that it comes to the actual eye, nose, mouth, and jaw position. Specifically, the reference coordinates of the pupil, nose, mouth, and jaw are compared with the average nose, mouth, and jaw position ratios of the general people based on the reference coordinates of the pupil extracted using the face detector, Since the reference coordinates are generated, the face detector operator can track the wrong pupil due to the low-quality image or the dark image, and the position of the nose, mouth and jaw of the face image can be detected from the average nose, (Step S220) to fine-adjust the contour coordinates to the eyes, nose, mouth, and jaw of the actual face image. Therefore, it is needless to say that the fine adjustment step S220 may be omitted if the eye, nose, mouth, and jaw of the actual image coincide with the reference coordinates of the eyes, nose, mouth, and jaw.

As a method of moving the reference coordinates, an example is described in which a reference ball is moved and input using a track ball. However, various methods of movement such as drag and drop or direct input of coordinates are used And it goes without saying that the present invention also belongs to the present invention.

9, the outline of the eyes, nose, mouth, and jaw is generated based on the reference coordinates of the pupil, nose, mouth, and jaw. In the reference coordinates of the extracted pupil, nose, mouth, (S230) of creating contours of eyes, nose, mouth, and jaw in contrast with the ratio of sizes pre-stored in the storage unit of the server or the server. In detail, the pre-stored size ratio stores size ratio information, which is information on each size of eyes, nose, mouth, and jaw and distance between eyes, nose, mouth and jaws. , Contour lines are generated by comparing the size ratios to the reference coordinates of the mouth and jaw. More specifically, for example, when the distance between the two reference coordinates of the eyes located in the two eyes is 20, the distance between the two eyes is 10, and the size ratio information stored as 1 is collated , The distance between the two eyes is doubled, so the eye size is multiplied by two. Therefore, since the size of the eye is 2, a contour having a size of 2 is generated.

Next, referring to FIG. 9, at least one outline coordinate located on a contour line is generated (S240) based on the contour line generated in the face image through the above process. More specifically, at least one outline coordinate located on the outline is a point for utilizing mesh warping in which the image is distorted (formed) by moving the outline coordinates in a step S240 described later, and the outline coordinates are formed on the outline . In addition, the outline coordinates located on the contour line can be generated by varying the number of the outline coordinates according to a preset value of the user. For example, if a user creates six outline coordinates in the eye and inputs a set value that creates 10 outline coordinates in the nose, the eye connected to one end of the outline is divided into six equal parts, One outline coordinate is generated at one end and the other end of the outline in the case of a nose into which 10 outline coordinates are inputted and the remaining eight outline coordinates are generated by being positioned on the outline at a predetermined interval. In this case, the outline coordinates located on the outline are generated by preliminarily inputting the number. However, the outline coordinates can be generated by presetting the distance between the outline coordinates as well as the number, and this also belongs to the present invention .

On the other hand, a step S230 of creating an eye, a nose, an mouth, and a jaw contour based on the reference coordinates of the generated pupil, nose, mouth, and jaw and a step S230 of creating at least one Although the step S240 of generating outline coordinates has been described by dividing the two steps, two steps can be simultaneously performed. That is, the process of generating the outline and the process of generating the outline coordinates located on the generated outline can be performed at the same time, and the present invention also belongs to the present invention.

Referring to FIG. 10, after generating at least one outline coordinate located on the outline of the eyes, nose, mouth, and jaw through the above process (S240), the outline coordinates are calculated based on the actual eye, nose, (S250) for fine adjustment to be positioned on the contour line. In detail, the outline coordinates are located on the outline, and the outline is determined based on the average eye, nose, mouth, and jaw size ratios of people stored in advance in the storage unit by the user as described in the above process Thus, the generated contour may be different from the actual eye, nose, mouth, and jaw contour of the facial image. To correct this, the contour and contour coordinates may be located on the contour of the actual eye, nose, (S250). It is needless to say that, in the step of finely adjusting the outline coordinates, when the outline coordinates and contour lines generated as described above coincide with the actual eye, nose, mouth, and jaw contour of the face image, the fine adjustment step can be omitted .

At this time, as an example of moving the outline coordinates, the outline coordinates are moved and input using a track ball as an example. However, various methods of movement such as drag and drop or direct input of coordinates And it goes without saying that the present invention also belongs to the present invention.

Here is an explanation of the step of generating outline coordinates located on the outline of the eyes, nose, mouth and jaw in the face image.

Next, the mesh generation and mesh group generation step before the image is transformed using the mesh warping is a task executed by the processor, and does not output a separate interface and a screen to the user, and therefore, the drawing is omitted.

Referring to FIG. 1, after generating outline coordinates (S100) through the above process, a step S110 of generating a plurality of mesh points including outline coordinates in a face image and arranged at predetermined intervals is performed. In detail, a mesh point is a preparation step for a step for creating a mesh group necessary for deforming an image by using a mesh warping which will be described later, the mesh point being composed of an array of point grid shape spaced apart by a predetermined distance, Is provided including outline coordinates. The mesh point can be generated by a preset value by the user. That is, it can be generated in accordance with conditions such as a distance, a number and a coordinate set in advance by the user.

Next, the step of creating a mesh group using the generated mesh point (S120), a step 120 of creating a mesh group using the mesh point includes a process of generating a mesh having the mesh point as a vertex . In detail, the mesh may be provided as a triangular mesh that is generated by connecting three mesh points. The mesh group used in general mesh warping is composed of a mesh group consisting of a square mesh. The mesh of the present invention is provided as a triangular mesh generated by connecting three mesh points. Although the invention is described by way of example in which a mesh is created as a triangular mesh connecting three mesh points, it is not necessarily limited to a triangular mesh with three mesh points connected, but a square mesh of four mesh points, A pentagonal mesh having five mesh points, and the like, and it goes without saying that the present invention also belongs to the present invention. However, as in the present invention, when the three mesh points are formed as a set of triangular meshes connected to each other, the mesh can be more clearly and finely manipulated than the conventional square mesh, and smooth output can be obtained.

Next, referring to FIG. 11, there is a step (S130) of transforming the image using mesh warping to the mesh group generated in the above process (S130). The mesh group generated in detail is generated as a mesh point including outline coordinates In this step, the image is distorted / deformed by shifting the outline coordinates by the coordinates desired by the user. According to the present invention, since the mesh group consisting of triangular meshes is mesh warped, sharp, detailed manipulation is possible and soft output can be obtained as compared with the conventional case of meshing a group of meshes made of a quadrangle. Particularly, the present invention of mesh-warping a face image of a person is characterized in that the effect is superior to mesh-warping of a square mesh when the mesh group consisting of a triangular mesh is meshed.

Here, the "mesh warping" means that a new point is selected as an arbitrary point in the image and a control point is moved to create a new image. It is a technology developed to solve distorted images output from the initial magnifying glass, and is a technique mainly used for various special effects such as movies. In detail, mesh warping is a process of converting a pixel located at coordinate values of x and y of an image into an image (Ux (x, y), Uy (x, y) x, y) to transform the image. In other words, mesh warping changes the shape of the mesh that makes up image A by selecting point P as the control point in the image and moving it to point P '. We transform each mesh of the original image A with P control points into an affine transform and transform it into a new image A 'with a control point. Since the mesh warping is a known technology, a detailed description will be omitted.

In the method of applying mesh warping, mesh warping is performed by moving the outline coordinates. Although the movement and the input of the coordinates and the coordinates by using a track ball are described as an example, the drag and drop ) Or by directly inputting coordinates, and it goes without saying that the present invention also belongs to the present invention.

The image modified by the mesh warping may have a post-processing process on the image modified by the interpolation method. The image interpolation method is a linear filtering method to solve heterogeneous feeling of pixels (texels, texels) of a transformed image. By performing linear interpolation on arbitrary texels and the midpoints of texels adjacent to the texels in four directions, Reduce the image will look smooth. Specifically, when creating / transforming a texture-covered object, the texture is not output as it is stored. When you zoom in or out of a texture or convert a viewpoint, pixels on the texture (texel, texel) will pop out. Linear filtering performs linear interpolation on the center points of arbitrary texels in a texture and adjacent texels in four directions to make the texture look smooth in any case.

For example, in the following equation, uk and vk are the coordinates of the texture, k is the point, and k is the color. Values without subscripts represent points of pixels, values with subscripts 0, 1, 2, and 3 represent texels of the top, left, right, and bottom adjacent to each pixel, and assuming the texture is a square bitmap,

The equations in Equation (1) are true. By defining the following expression,

The interpolation equation can be simplified as shown in Equation (2).

Then, the equation (3) is combined to obtain the following equation (4).

Alternatively, Equation (5) can be expressed differently.

Referring to FIG. 11, in step S130 of deforming and forming a facial image using mesh warping, the facial image may be further formed with a pair of eyes on the eye of the facial image. Specifically, it may further have a step of creating a line along the double-skinned line at the coordinates where the double skin of the upper part of the eye is located. As an example, when generating a double-skinned balloon, a line is generated at a coordinate at which the double-skinned balloon is positioned as described above. However, there is a method of generating a double-skinned balloon using mesh warping at the coordinates where the double- .

Referring to FIG. 12, the facial image is formed through the above-described process, and the formed image is outputted through the output unit of the terminal. The step of outputting the formed image can be divided into three types. Three methods of outputting the formed image include a method of immediately outputting the formed image to the output unit as it is, a method of outputting the molded image in three dimensions Methods (S160, S170), and a method (S140, S150) for outputting a molded image together with a golden ratio of a face for comparison. The following three output methods will be described in detail.

After shaping the image through mesh warping, the molded image is basically output through the output without additional processing. Thereafter, the user may be selectively divided into a comparison with the golden ratio (S150), or a stereoscopic output step (S170) in which the stereoscopic output is performed.

Referring to FIG. 13, the golden ratio step includes extracting and outputting a ratio and a coordinate between the outline or outline coordinates of the eye, nose, mouth, and jaw of the formed image and outputting the calculated ratio and outputting a golden ratio to the extracted ratio and coordinates (S150). In detail, the step S140 of outputting the relative ratios and coordinates between the outline of the eye, nose, mouth and jaw of the image or coordinates of the outline of the image may include a distance and a relative ratio between eyes, nose, mouth and jaw of the formed image, The relative ratio and size of the nose, mouth, and jaw are extracted and a numerical value and a guide line are output on the formed image (S150), whereby the relative ratio and size of the eyes, nose, . At this time, the outline or outline coordinates of the eyes, nose, mouth, and jaw, which are used for measuring the relative ratio and size, may be determined based on the relative ratio and size of the outline or contour coordinates, Is measured. In addition, the golden ratio is a relative ratio and coordinates of eye, nose, mouth, and jaw of a beautiful face stored in advance in the storage unit, and the golden ratio of the beauty face is also output on the formed image. That is, since the relative proportions of the eye, nose, mouth, and jaw of the formed image and the relative ratios and coordinates of the eye, nose, mouth, and jaw of the coordinate system and the beautiful face are output together on the formed image, Since it can be compared and compared with the face, it is guided to revise or evaluate the image, and it is characterized that it can grasp the difference between the formed image and the beauty face. The facial image of his or her own face usually has complexity, which is not evaluated by another person or an expert but is advantageous in that it can be compared / judged by comparing with an objective beauty face. At this time, although the relative ratios and coordinates of the eyes, nose, mouth, and jaw of the beauty face are previously stored in the storage unit of the terminal, the beauty face stored in the server can be used through the communication unit 15, This also belongs to the present invention. In addition, although the golden ratio of the beauty face is exemplified as being the golden ratio of the beauty face stored in advance by the user, it may be a face image selected by the user. In other words, it can be compared with the face ratio of the face image selected by the user as well as the comparison / comparison with the golden ratio of the face image. The face image selected by the user is extracted by extracting the relative ratios and coordinates of the eyes, nose, mouth, and jaw of the selected face image in the same manner as the method of extracting the relative ratios and coordinates of the eyes, nose, mouth, It is preferable to output it. In the above process, it is desirable that the relative ratio of the formed image and the relative ratio and coordinates of the coordinate of the golden face of the beautiful face are generated on the same basis.

Also, the step of outputting and comparing / comparing the relative ratio of the formed image and the golden ratio of the coordinate system and the beauty face may be performed in the mesh warping step of forming the image. In detail, when the image is formed by using the mesh ratio of the golden ratio of the beauty face, the golden ratio of the beauty face is output, so that when the user shapes the image using the mesh warping, So that a more effective forming guide can be made.

14 and 15, the stereoscopic output step (S160, S170) during the step of outputting the molded image applies a unique rotation weight corresponding to each direction of the upper, lower, left, and right for each mesh point of the deformed image (S160) And three-dimensionally 3Dized and outputted (S170). This gives an illusion effect as if a face image of a plane (2D) is viewed as a three-dimensional object (3D). By moving each mesh point by a previously stored value using mesh warping, In this case, it is possible to output an image that is rotated by 90 °. However, in order to realize realism, an image is output by 3Dizing the angle of 15 ° to the maximum value. However, it is not necessarily limited to the angle of 15 degrees, but may be output at various angles, and it goes without saying that the present invention also belongs to the present invention. A specific rotation weight corresponding to each direction of the upper, lower, left, and right sides is applied to each mesh point to output in three dimensions.

(S160), the coordinates of the mesh point are shifted, and mesh warping is applied to the moved mesh points to apply the image to the mesh points located on the formed image (S170). When the above process is performed, the 2D image brings an optical illusion that looks like 3D. More specifically, the user has an interface for rotating the image in the left, right, upper, and lower directions. When a command to rotate the image at a predetermined angle is input to the interface, A pre-stored moving weight corresponding to the angle is applied to the mesh point so that the mesh point is shifted by the corresponding moving weight value to mesh-warp.

For example, when the user attempts to rotate the image to the right to see the side of the nose after shaping the nose, the mesh points corresponding to the nose are set such that the mesh point against the left contour of the nose is a1, A2 and a3 corresponding to the right contour line of the nose are formed as a3, and when the user rotates the image to 15 degrees to the right, a1, a2 and a3 are moved by the previously stored coordinate movement weight values, a1 has a coordinate shift weight value of (0, 0) corresponding to a rotation of 15 degrees to the right, a2 has a coordinate shift weight value of (3, 0) corresponding to a rotation of 15 degrees to the right, (0, 0) corresponding to the rotation of 15 degrees, when the image is rotated by 15 degrees to the right, a1 corresponds to the rotation of 15 degrees to the right (0, 0) coordinate movement A2 is shifted to the right side of (x + 3, y + 0) because it has the coordinate shift weight value of (3, 0) (X-1, y + 0) because it has the coordinate shift weight value of 0 (zero). The image is then warped to the coordinate values of the moved a1, a2 and a3, and the image is transformed. As a result, the image between a1 and a2 undergoes an elongated transformation process, and the image between a2 and a3 undergoes a deformation process, resulting in a stereoscopic effect such as looking at the actual nose from the left side, It is possible to obtain the same optical illusion effect as that of viewing. At this time, it is desirable that the mesh point formed at the edge of the face image is not moved so that the 2D image is more effectively 3Dized through stereoscopic molding.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

10: Terminal 11: Processor
12: storage unit 13: output unit
14: input unit 15:
20: Server

Claims (19)

A method for forming a virtual face using mesh warping in a terminal,
(S100) generating outline coordinates located on the outline of the eyes, nose, mouth, and jaw in the face photographed image;
Generating (S110) a plurality of mesh points including the contour coordinates in the image and arranged at a predetermined spacing;
Creating a mesh group using the mesh point (S120);
Forming the image using mesh warping on the mesh group (S130); And
And outputting the molded image,
The step of generating the outline coordinates (SlOO)
Generating reference coordinates of the pupil in the image (S200);
(S210) generating reference coordinates of the nose, mouth, and jaw by comparing the reference pupil coordinates of the generated pupil with the previously stored common pupil, nose, mouth, and jaw ratio;
(S220) finely adjusting the reference coordinates of the pupil, nose, mouth, and jaw to be located in the pupil, nose, mouth, and jaw of the image;
(S230) a contour of eyes, nose, mouth, and jaw in accordance with a ratio of sizes of the pupil, nose, mouth, and jaw to the reference coordinates of the pupil, nose, mouth, and jaw;
Creating (S240) at least one contour coordinate located on the contour of the eyes, nose, mouth and jaw; And
(S250) fine-adjusting the outline coordinates of the eye, nose, mouth and jaw to be located on the contours of the eyes, nose, mouth and jaw of the image (S250) Gt;
delete delete delete The method according to claim 1,
The step S200 of generating the reference coordinates of the pupil in the image
Wherein a reference coordinate of a pupil is generated in the image using a face detector operator.
delete delete The method according to claim 1,
The step (S120) of creating a mesh group using the mesh point,
Further comprising generating a mesh having the mesh point as an apex,
Wherein the mesh is a set of triangular meshes generated by connecting three mesh points.
The method according to claim 1,
(S130) forming the image using mesh warping on the mesh group,
Wherein the mesh group is mesh warped by moving the outline coordinates. ≪ RTI ID = 0.0 > 11. < / RTI >
The method according to claim 1,
(S130) forming the image using mesh warping on the mesh group,
(S140) extracting and outputting a ratio and a coordinate between an outline of the eye, nose, mouth, and jaw of the image or outline coordinates; And
And outputting the golden ratio to the extracted ratio and coordinates and comparing (S150)
Wherein the golden ratio is a relative ratio, an angle, and a coordinate of eyes, nose, mouth, and jaw of a previously stored beautiful face.
The method according to claim 1,
After outputting the molded image,
(S140) extracting and outputting a ratio and a coordinate between an outline of the eye, nose, mouth, and jaw of the image or outline coordinates; And
And outputting the golden ratio to the extracted ratio and coordinates and comparing (S150)
Wherein the golden ratio is a relative ratio, an angle, and a coordinate of eyes, nose, mouth, and jaw of a previously stored beautiful face.
The method according to claim 1,
After outputting the molded image,
And a stereoscopic output step (S 170) of stereoscopically outputting the formed image,
The stereoscopic output step of stereoscopically outputting the formed image includes:
Wherein the molded image is 3Dized and output so as to be rotatable by a predetermined angle.
13. The method of claim 12,
The stereoscopic output step of stereoscopically outputting the formed image includes:
And a unique moving weight corresponding to each direction of vertically, horizontally, and vertically stored in advance for each mesh point of the formed image is applied, and the mesh warping is output.
A terminal (10) for implementing virtual plastic surgery,
Creating a plurality of mesh points arranged at predetermined intervals, generating contour coordinates located on the contour lines of the eyes, nose, mouth and jaw in the facial photographing image, including the contour coordinates in the image, A processor (11) for creating a mesh group using points and shaping the image using mesh warping to the mesh group;
An output unit 13 for outputting the formed image;
A storage unit (12) in which at least one of a position ratio of a face image, a pupil, a nose, a mouth and a jaw, a size ratio and a relative ratio of an eye, a nose, a mouth and a jaw, And
And an input unit (14) for receiving the fine adjustment value,
The processor (11), in generating the contour coordinates,
Generating reference coordinates of the pupil in the image,
The reference coordinates of the pupil, the mouth, and the jaw are generated by collating the pupil, nose, mouth, and jaw position ratios stored in the storage unit with the reference coordinates of the pupil,
Adjusts the reference coordinates of the pupil, nose, mouth, and jaw to be located in the pupil, nose, mouth, and jaw of the image based on the fine adjustment value input through the input unit (14)
Wherein the controller compares the reference pupil, nose, mouth, and jaw reference coordinates of the generated pupil, nose, mouth, and jaw with the ratio of pupil, nose, mouth, and jaw sizes stored in the storage unit,
Generating at least one outline coordinate located on the contour of the eye, nose, mouth, and jaw,
The fine coordinates are adjusted so that the outline coordinates of the eyes, nose, mouth and jaw are located on the outline of the eyes, nose, mouth and jaw of the image based on the fine adjustment value inputted through the input unit 14. [ A terminal that implements virtual plastic surgery using mesh warping.

delete delete delete delete 15. The method of claim 14,
The output unit extracts and outputs a ratio and a coordinate between an outline or an outline coordinate of an eye, nose, mouth, and jaw of the image, outputs a golden ratio to the extracted ratio and coordinates,
Wherein the golden ratio is a relative ratio, an angle, and a coordinate of eyes, nose, mouth, and jaw of the beauty face stored in the storage unit.

Images