KR101839515B1 - Service systembased on face recognition inference, and face recognition inference method and storage medium thereof - Google Patents

Abstract

The present invention relates to a face recognition inference based service system and method for inferring side and rear images from a flat face image and providing a more stereoscopic and realistic virtual styling service to the user based on the images, A method for extracting feature information of a face from a face image of a user, extracting type information of a face from facial feature information, combining facial feature information and type information of a face, The virtual face modeling image is combined with at least one of the front face, the side face, and the back face face image of the user to provide the virtual beauty modeling image to the user, Provide services based on reasoning do.

Description

[0001] The present invention relates to a face recognition inference service based system, a face recognition inference method, and a storage medium,

The present invention relates to a face recognition inference based service system and method for inferring side and rear images from a flat face front image and providing a stereoscopic and realistic virtual styling service to the user based on the images, ≪ / RTI >

In recent years, as a smartphone that users can freely select and install a desired mobile application is rapidly growing, OMP (Open Market Place) that can freely deal with such a mobile application is being activated. Accordingly, development of various application programs and services .

As one of them, a virtual hair styling service is provided which allows a virtual hair to be changed.

However, since the existing virtual hair styling service captures only the front image of the user and provides the virtual hair styling for the front image, the user can not check the side and rear views.

In order to service the side or rear view, it is necessary to shoot the image in various directions of the user's image and then combine them to generate a three-dimensional image of the user. In this case, since expensive equipment must be provided, There is a problem that occurs.

Therefore, research is needed to provide various virtual hair experience services to users in a less expensive way without using expensive equipment.

Korean Patent Laid-Open No. 10-2009-0073294, published on Jul. 03, 2009 (name: method of inferring human interrelationships in images through face recognition)

Accordingly, the present invention provides a face recognition reasoning based service system and method for inferring side and rear images from a flat face front image and providing a stereoscopic and realistic virtual hair styling service to a user based on the images, And a face recognition inference method.

As a means for solving the above-mentioned problems, the present invention is a method for extracting feature information of a face from a front face image of a user, extracting type information of a face from face feature information, combining feature information of a face, A service device for generating a virtual beauty modeling image by combining at least one of a facial image of a specific style and a front face, a side face, and a back face image of a user by inferring at least one of face and back face images of a user; And a terminal device for providing a front face image of a user to a service device, receiving the virtual beauty modeling image from the service device, and outputting the virtual beauty modeling image to a user.

In addition, the present invention provides, as another means for solving the above-mentioned problems, a communication device for transmitting / receiving data to / from a terminal device via a network; And a communication unit for receiving a front face image of the user from the terminal device, extracting face feature information from the received front face image, extracting face type information by comparing and analyzing facial feature information, And a face recognition unit for combining at least one face image of the user by combining the type information of the face.

The service apparatus according to the present invention may combine at least one of front, side, and rear face images of a user with specific beauty objects to generate a virtual beauty modeling image expressed in at least one direction from the front, To the virtual beauty service section.

The service apparatus according to the present invention includes a face type DB for storing information on a plurality of face types compared with face feature information in a face recognition unit and divided into at least one of face type, race, gender, and age; A facial image DB storing side and back facial images deduced from facial feature information and facial type information in a facial recognition unit; And a beauty object DB for storing a plurality of beauty objects to be used in the virtual beauty service unit.

According to another aspect of the present invention, there is provided an information processing apparatus including an input unit for generating a user input signal according to a user operation; A camera unit for photographing a front face image of a user; Extracts feature information of face from front face image, extracts type information of face by comparing and analyzing facial feature information, combines feature information of face and type information of face to infer one or more of side and back face images A front face image of the user is obtained by controlling the camera unit according to the input signal of the user, and one or more of the user's front and rear face images are acquired from the front face image of the user acquired through the face recognition module ; And an output unit for outputting at least one of a front face image, a side face and a rear face image of the user according to the control of the control unit.

In the terminal device according to the present invention, the control unit may further include a virtual modeling module for generating a virtual beauty modeling image by combining at least one of front, side, and back face images of a user with a cosmetic object selected by the user.

The terminal device according to the present invention may combine a plurality of face type information, facial feature information, and face type information to be compared with facial feature information, which is divided into at least one of a front face image, a face form, a race, An inferred side and back face image, and a plurality of beauty objects.

The terminal device according to the present invention may further include a communication unit for transmitting and receiving data through a network, and may receive data for inference or virtual beauty modeling creation from at least one of the side and back face images from the service device.

In addition, according to another aspect of the present invention, there is provided a method for processing a face image, comprising: obtaining a face image of a user; Extracting feature information of a face from a front face image; Comparing the feature information of the extracted face with the pre-classified type, and extracting type information of the face; And combining the feature information of the extracted face and the type information of the face to generate a polygon for the front face and generating a side face and a back face image of the user from the polygon, do.

In the facial recognition reasoning method according to the present invention, the step of extracting facial feature information includes extracting eyes and mouth from a front face image, Extracting the nose region by connecting the extracted eyes and mouths in an inverted triangle shape; Extracting detailed information of the nose by analyzing edge and color changes within the nose region; Extracting an eyebrow by analyzing a color change in an area on the extracted eye; Extracting feature points and contour lines of a face; And extracting facial feature information of the user necessary for polygon generation by combining measured values and statistical information for each extracted facial region.

In the face recognition reasoning method according to the present invention, the face type information may be classified into at least one of face type, race, sex, and age.

In the face recognition inferring method according to the present invention, the face feature information includes at least one of an overall length and a width of a face, a length of a jaw, a height of a nose, , And the height of the forehead.

In addition, according to another aspect of the present invention, there is provided a method for processing a face image, comprising: obtaining a face image of a user; Extracting feature information of a face from a front face image; Comparing the feature information of the extracted face with the pre-classified type, and extracting type information of the face; Combining the feature information of the extracted face and the type information of the face to generate a polygon for the front face and generating at least one of the side and back face images of the user from the polygon; Receiving selection information of a beauty object that the user wishes to experience; A step of generating a virtual modeling image represented by at least one of front, side, and rear directions of a user by combining the user's selected hair object with at least one of a user's front face image, a side face image, and a rear face image, There is provided a computer readable recording medium recording a program to be executed.

According to the present invention, by inferring one or more of a user's side face image and a rear face image with only a two-dimensional face image of a user photographed through a general camera, a more stereoscopic and realistic user image can be realized without expensive equipment, In addition, various application services based on face recognition can be provided based on the side face image and / or the rear face image deduced from the user's front face image.

In addition, according to the present invention, the side face image and the rear face image inferred from the user's front face image are applied to various virtual beauty experience services including hair and makeup, It is possible to provide an enhanced virtual beauty experience service, and the service provider has an excellent effect of reducing the cost for implementing the virtual beauty experience service.

1 is a block diagram illustrating a face recognition reasoning based service system according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of a service apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing a face recognition inference service in a service apparatus according to an embodiment of the present invention.
4 is a block diagram illustrating a configuration of a terminal apparatus according to another embodiment of the present invention.
5 is a flowchart illustrating a method of providing a face recognition inference service in a terminal device according to another embodiment of the present invention.
FIG. 6 is a flowchart illustrating a detailed process of extracting feature information in the face recognition inference method applied to the service method according to the present invention.
FIG. 7 is an exemplary diagram illustrating a process of extracting facial feature information from a facial facial image in the facial recognition inference method according to the present invention.
FIG. 8 is an exemplary diagram for explaining a process of inferring side and back face images from a front face image in the face recognition inference method according to the present invention.
FIG. 9 is a diagram illustrating an example of a face recognition inference based service method according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

1 is a block diagram illustrating a face recognition reasoning based service system according to an embodiment of the present invention.

Referring to FIG. 1, a face recognition reasoning based service system according to the present invention includes a service device 100 and a terminal device 200 connected via a network 10.

Here, the service apparatus 100 extracts feature information of a face from a front face image of a user by applying the face recognition inference method according to the present invention, extracts type information of the face from the face feature information, Feature information and type information of the face are combined to infer one or more of the user's side and back face images. For this, the service apparatus 100 can receive the front face image of the user from the terminal apparatus 200 through the network 10 and receive, as the face recognition result from the terminal apparatus 200, one Or more. Here, the process of inferring the side and back face images from the front face image of the user can be performed as shown in FIG. 6, and a description thereof will be described later in more detail.

In addition, the service apparatus 100 can use the front face image of the user and the side and back face images of the user generated through inference, for any application service based on face recognition. For example, the service device 100 may combine a user's image with a hair or makeup related beauty object to provide a virtual beauty experience service to the user. At this time, the service apparatus 100 can provide the virtual beauty modeling image for the front side, the side and the rear side to the user by using the side face and the rear face image deduced from the front face image of the user. That is, it is possible to provide more stereoscopic and realistic virtual beauty services only by using a flat front face image without using expensive equipment.

At this time, the terminal device 200 is a device used by a user who wants to use the face recognition based service, and connects to the service device 100 through the network 10 to transmit the face image of the user to the service device 100. [ And receives the virtual beauty modeling image from the service device 100 and outputs the virtual beauty modeling image to a user.

In the above-described facial recognition inference-based service system, the service apparatus 100 may operate in a server-client computing mode or a cloud computing-based mode. For example, the service device 100 may be implemented as one or more servers operating on a cloud computing basis.

The type of the network 10 connecting the service device 100 and the terminal device 200 is not particularly limited. For example, the network 10 is an i-bimetry that provides a large capacity data transmission / reception service and a data service without a disconnection phenomenon through an internet protocol (IP) It may be an All IP network. Also, the network 10 may include a 3G mobile communication network including a wired network, a Wibro (Wireless Broadband) network, a WCDMA, a 3.5G mobile communication network including an HSDPA (High Speed Downlink Packet Access) A fourth generation mobile communication network, a satellite communication network, and a Wi-Fi network.

Also, the terminal device 200 can be used as any communication device available to the user. For example, the terminal device 200 may be a personal computer (PC), a notebook computer, a mobile phone, a tablet PC, a navigation terminal, a smart phone, a PDA (personal digital assistant) Portable Multimedia Player) and a digital broadcast receiver. Of course, this is merely an example, and it should be construed as a concept including a device that is currently developed, commercialized, or capable of all communication to be developed in the future, in addition to the above-described examples.

Next, FIG. 2 is a block diagram showing a configuration of the service apparatus 100 according to an embodiment of the present invention, which is divided into functional units.

2, the service apparatus 100 according to the present invention includes a communication unit 110, a face recognition unit 120, a virtual beauty service unit 130, and a storage unit 140.

The communication unit 110 is a means for transmitting and receiving data through the network 10, and more specifically, transmits and receives data to and from a terminal device 200 of a user who is provided with a face recognition service.

The face recognition unit 120 receives the user's front face image from the terminal device 200 through the communication unit 110, extracts the face feature information from the received front face image, Extracts the face type information, combines the feature information of the face and the type information of the face to infer one or more of the user's side and back face images.

The face recognition unit 120 may infer the side and back face images from the front face image of the user according to the face recognition method shown in FIG. The face recognition unit 120 may provide the side and back face images deduced from the front face image of the user to the terminal device 200 according to a user's request.

The face recognition unit 120 may include a feature extraction module 121, a type extraction module 122, and an image reasoning module 123.

The feature extraction module 121 extracts feature information of a face representing a feature of each part of the face (eye, nose, mouth, eyebrow, outline, forehead, chin, etc.) from the user's front face image. More specifically, the feature extraction module 121 first extracts eyes and mouth from the user's frontal face image. Here, the eye extraction may be performed by first extracting a pupil, and then extracting an eye shape based on the pupil. In addition, such eye and mouth extraction can be performed using general face recognition technology. Then, the feature extraction module 121 extracts the nose region by connecting the extracted eye and mouth in an inverted triangle shape, extracts detailed information of the nose by analyzing the edge and color change in the nose region, Extracts eyebrows by analyzing the color change in the above area, and extracts the feature points and contour lines of the face, and extracts detailed information of each part of the face. Then, the measured values and statistical information And extracts feature information for each face region of the user. Here, the feature information of the face includes information on the front face of the user such as the total length and width of the face, the length of the chin, the height of the lower part, the center of the nose, the height of the nose, the length of the nose, It contains various information needed for polygon generation.

Next, the type extraction module 122 compares the feature information extracted from the feature extraction module 121 with the pre-classified type, and extracts the type information of the face matched with the user's face. Here, the type information of the face is obtained by dividing the faces of various users according to certain criteria. For example, the type information (egg type, round type, four type, and three type), race (yellow, white, black, , Sex (male, female), age (infancy, childhood, adolescence, adulthood, old age, etc.).

Finally, the image reasoning module 123 combines the feature information of the face extracted from the feature extraction module 121 and the type extraction module 122 with the face type information to generate a polygon for the front face of the user And infer the side and back face images of the user therefrom. At this time, the learned face image can be referred to. That is, it is possible to refer to side and back face images previously deduced or preset for the same face type. At this time, the side and back face images generated by the image reasoning module 122 may be three-dimensional or two-dimensional still images. In particular, the side face images may be further subdivided based on a certain angle. For example, when the front surface is a reference point, a first side face image of 0 to 60 degrees, a second side face image of 61 to 90 degrees, and a third side image of 90 to 140 degrees can be generated.

Then, the virtual beauty service unit 130 provides a virtual experience service related to the cosmetic field including hair and makeup through the network 10. More specifically, the virtual beauty service unit 130 includes a front face image of the received user, One or more of the facial images generated through inference from the facial recognition unit 120 may be combined with a specific beauty object to generate a virtual beauty modeling image expressed in one or more directions from the front, To the device (200). For example, the virtual beauty service unit 130 may combine a hair object having a specific style shape with one or more of the front, back, and side face images of the user to change the hair style of the hair object (Front, side, and rear) of the appearance of the image when it is made.

The storage unit 140 stores data necessary for performing face recognition inference and virtual beauty experience service in the face recognition reasoning based service device 100 according to the present invention. More specifically, the storage unit 140 stores information on a plurality of face types that are compared with the face feature information in the face recognizing unit 120 and are divided into at least one of face type, race, gender, and age A facial image DB for storing facial type DB stored in the facial image recognition unit 120, facial image DB for storing side and back facial images deduced from facial characteristic information and facial type information in the facial recognition unit 120, And provides the requested information in response to a request from the face recognition unit 120 and the virtual beauty service unit 130. [ The storage unit 140 may be a RAM, a read only memory (ROM), a hard disk drive (HDD), a flash memory, a CD-ROM, NAS, and Network Access Storage).

3 is a message flow diagram illustrating a method of providing a face recognition inference service through the service device 100 according to an embodiment of the present invention.

Referring to FIG. 3, the terminal device 200 accesses the service device 100 according to a user input signal and requests a service (S105).

Then, the service apparatus 100 confirms whether the user is allowed to use the service through the user authentication, and then transmits a response to the terminal apparatus 200 to permit the service (S110).

After that, the terminal device 200 acquires the front face image of the user and transmits it to the service device 100 (S115, S120). This is because the terminal apparatus 200 operates the camera unit 230 in response to a request from the service apparatus 100 to photograph a front face image of a user or to display a front face image of a user stored in the storage unit 260 And transmits the data to the service apparatus 100. [

Then, the service apparatus 100 executes the face recognition process for the front face image of the user. First, the service apparatus 100 receives the face image from the received front face image through the feature extraction module 121 of the face recognition unit 120, (S125). Here, the feature information of the face means feature information of a face that shows the feature of each part of the face (eye, nose, mouth, eyebrow, outline, forehead, jaw, etc.) from the front face image of the user, The various information needed to create a polygon for a user's face, such as the total length and width of the face, the length of the chin, the height of the nose, the center of the nose, the height of the nose, the length of the nose, the height of the mouth, the length of the mouth, . The feature information of the face can be extracted through a process as shown in FIG. 6, and FIG. 7 illustrates a front face image of the user processed through the process of FIG. In FIG. 7, each point on the facial image is a feature point representing each part of the face extracted from the front face image of the user, and each of the facial parts can be expressed by connecting these feature points.

Next, the face recognition unit 120 of the face recognition-based service apparatus 100 compares the extracted feature information of the face with the pre-classified type through the type extraction module 122 to extract the type information of the face (S130). Here, the type information of the face is obtained by dividing the faces of various users according to certain criteria. For example, the type information (egg type, round type, four type, and three type), race (yellow, white, black, , Sex (male, female), age (infancy, childhood, adolescence, adulthood, old age, etc.). The step (S130) compares the predetermined standard information for each face type with the feature information of the extracted user face, and extracts the user's face type information according to the matching result.

The face recognition unit 120 of the service apparatus 100 combines the feature information of the extracted face with the type information through the image inference module 123 to generate a polygon for the front face image of the user, At least one of the side and back face images of the user is generated through inference (S135). The inference of the user's side and back face images can be done with reference to the learned face images. For example, after generating a polygon for the front face image of the user, the polygon is rotated at a specified angle, and the side image and the back image of the standard face of the face type extracted from the learned face image are combined have. FIG. 8 illustrates a polygon of a front face image of a user generated from a user front face image as shown in FIG. 7 and a polygon of a side face image deduced therefrom. 8 (a) shows a polygon generated from a front face image, and FIG. 8 (b) shows a polygon derived from a side face image by rotating the polygon by a predetermined angle. Here, the service apparatus 100 may provide at least one of the speculated side and rear face images to the terminal device 200. [

Then, the service apparatus 100 executes the face recognition-based service using the above-described face recognition result. To this end, the service apparatus 100 requests the terminal device 200 to select a beauty object to be virtualized (S140) And receives a response including the object selection information (S145).

The service device 100, which has confirmed the beauty object to be experienced by the user through the response, combines the selected beauty object with at least one of the user's front face image and the generated side face and rear face image, The virtual beauty modeling image is represented by at least one of a front face, a side face, and a back face direction (S150), and the generated virtual beauty modeling image is transmitted to the terminal device 200 (S155).

The terminal device 200 outputs the received virtual beauty modeling image to the user (S160).

FIG. 9 illustrates an example of a face recognition based service provided through the above-described process. As shown in FIG. 9, the face recognition based service apparatus 100 according to the present invention extracts, from a user's front face image 91, It is possible to provide both the side modeling image 94 and the rear modeling image 95 as well as the front modeling image 93 of the user wearing the selected hair object 92 by inferring the side face image and the rear face image .

Meanwhile, in another embodiment of the present invention, the face recognition and face recognition based service can be performed mainly on the terminal device 200 without involvement of the face recognition service device 100 described above.

4 is a block diagram illustrating a configuration of a terminal device 200 according to another embodiment of the present invention.

4, the terminal 200 includes a communication unit 210, an input unit 220, a camera unit 230, a control unit 240, an output unit 250, and a storage unit 260 .

The communication unit 210 of the terminal device 200 communicates with the face recognition based service device 100 via the network 10 to perform face recognition and face recognition, And send and receive data related to face recognition based service execution.

The input unit 220 generates a user input signal according to a user's operation. The input unit 220 may be realized by various input means. For example, the input unit 220 may include at least one of a key input unit, a touch input unit, a gesture input unit, and a voice input unit. The key input means generates a signal corresponding to the key according to the key operation, and corresponds to a keypad and a keyboard. The touch input means is a touch pad, a touch screen, and a touch sensor, which recognize an input operation by sensing a user's operation of touching a specific portion. The gesture input means recognizes a specific operation, such as a shaking or moving operation of the terminal device, an approach to the terminal device, a blinking operation, etc., as a specific input signal, such as a geomagnetic sensor, A sensor, a camera, an altimeter, a gyro sensor, and a proximity sensor. The user can request the face recognition or face recognition based service through the input unit 220 or select specific beauty objects to be modeled for the face recognition based service.

The camera unit 230 is a means for photographing an image. In the present invention, the camera unit 230 can be used to photograph an image of a user for use in a face recognition and face recognition based service. In particular, the camera unit 230 photographs a two-dimensional frontal face image of the user.

The control unit 240 controls the entire operation of the terminal device 200 according to a user input signal generated through the input unit 220, and executes a specific function. The control unit 240 is a concept including a CPU or an MPU, and can perform various functions by operating according to an operating program and an application program. In the present invention, the controller 240 includes at least one of a face recognition module 241 and a virtual modeling module 242 for executing a face recognition reasoning and a face recognition reasoning based service.

The face recognition module 241 extracts feature information of the face from the front face image, extracts the type information of the face by comparing and analyzing the face feature information, and then combines the feature information of the face and the type information of the face, Infer one or more of the back facial images. The control unit 240 controls the camera unit 230 in accordance with the input signal of the user to acquire the front face image of the user and acquires the front face image of the user through the face recognition module 241 One or more of the user's side and back face images can be inferred and provided to the user.

The virtual modeling module 242 executes the face recognition based service using at least one of front, side, and back face images of the user. More specifically, the virtual modeling module 242 executes one of face, And combines the above-mentioned beauty objects selected by the user to generate a virtual beauty modeling image.

The output unit 250 outputs the operation status and operation result of the terminal device 200 to the user under the control of the controller 240. In particular, in the present invention, The virtual beauty modeling image generated from at least one of the front and back face images of the user and the front face, the side face, and the back face image of the user so that the user can recognize the virtual beauty modeling image. The output unit 250 may be implemented as a display unit such as an LCD (Liquid Crystal Display), a TFT-LCD (Thin Film Transistor-Liquid Crystal Display), an LED (Light Emitting Diodes) Light Emitting Diodes), AMOLED (Active Matrix Organic Light Emitting Diodes), flexible displays, and three-dimensional displays.

Finally, the storage unit 260 of the terminal device 200 stores data and programs necessary for the operation of the terminal device 200, and basically stores the operating programs and application programs to be executed by the control unit 240 do. In addition, the storage unit 260 stores data necessary for executing the face recognition reasoning and face recognition reasoning based services. For example, a plurality of face type information to be compared with the face feature information in the face recognition module 241 may be divided into at least one of a front face image, a face form, a race, a gender, and an age of the user, Face type information, and inferred side and back face images, and a plurality of beauty objects.

The terminal device 200 may capture the user's two-dimensional frontal face image and then provide one or more of the user's side and rear face images to the user on the basis of the photographed face image, And a rear face image based on at least one of the face image and the face image. At this time, the terminal device 200 receives data necessary for inferring one or more of the above-described side and back face images, or necessary for providing a face recognition based service, from the face recognition service device through the communication unit 210, 260).

5 is a flowchart illustrating a face recognition service method based on the terminal device 200 described above.

Referring to FIG. 5, the terminal device 200 first acquires a front face image of a user (S205). This can be done by operating the camera unit 230 and photographing the front face image of the user according to a user input signal generated through the input unit 220 by the terminal device 200. [ Also, this can be done by loading the front face image of the user stored in the storage unit 260 according to the user input signal.

When the front face image of the user is thus obtained, the terminal device 200 extracts the face feature information from the obtained front face image through the face recognition module 241 of the control unit 240 (S210). Here, the feature information of the face means feature information of a face that shows the feature of each part of the face (eye, nose, mouth, eyebrow, outline, forehead, jaw, etc.) from the front face image of the user, The various information needed to create a polygon for a user's face, such as the total length and width of the face, the length of the chin, the height of the nose, the center of the nose, the height of the nose, the length of the nose, the height of the mouth, the length of the mouth, . The feature information of the face can be extracted through the process as shown in FIG. 6, and FIG. 7 shows the front face image of the user processed through the process of FIG. In FIG. 7, a point displayed on the face image is a feature point representing each part of the face extracted from the front face image of the user, and the respective points of the face can be expressed by connecting the feature points.

Next, the terminal device 200 compares the extracted feature information of the face with the pre-classified type to extract face type information (S215). Here, the type information of the face is obtained by dividing the faces of various users according to certain criteria. For example, the type information (egg type, round type, four type, and three type), race (yellow, white, black, , Sex (male, female), age (infancy, childhood, adolescence, adulthood, old age, etc.). The step (S215) compares the predetermined standard information for each face type with the feature information of the extracted user face, and extracts the user's face type information according to the matching result.

Then, the terminal device 200 combines the feature information of the extracted face with the type information to generate a polygon for the user's front face image, and infer one or more of the user's side and back face images from the polygon S220). The inference of the user's side and back face images can be done with reference to the learned face images. For example, after generating a polygon for the front face image of the user, the polygon is rotated at a specified angle, and the side image and the back image of the standard face of the face type extracted from the learned face image are combined have. FIG. 8 illustrates a polygon of a front face image of a user generated from a user front face image as shown in FIG. 7 and a polygon of a side face image deduced therefrom. 8 (a) shows a polygon generated from a front face image, and FIG. 8 (b) shows a polygon derived from a side face image by rotating the polygon by a predetermined angle.

Then, the terminal device 200 executes the face recognition-based service using the above-described face recognition result. In response to the user input signal, the user selects a beauty object that the user wants to experience (S225). For example, you can select hair objects of a particular style.

Then, the terminal device 200 combines the selected beauty object with at least one of the front face image of the user and the generated side face and rear face image, respectively, and is expressed in at least one direction of the user's front face, side face, and back face The virtual beauty modeling image is generated (S230), and the generated virtual beauty modeling image is provided to the user through the output unit 250 (S235).

FIG. 9 shows an example of the face recognition inference service provided through steps S225 to S30. From the front face image 91 of the user of the present invention, the face image of the user and the face image of the back face It is possible to provide both the side modeling image 94 and the rear modeling image 95 as well as the front modeling image 93 of the user wearing the selected hair object 92. [

If necessary, the terminal device 200 may perform at least one of the operations for performing the face recognition reasoning and the face recognition reasoning based service execution, for example, steps 210 to S220, 100).

FIG. 6 is a flowchart illustrating a feature information extraction process in the face recognition inference method according to the present invention. The method described below can be applied to step S125 of FIG. 3 and step S210 of FIG.

A process of extracting feature information of a face from a user's front face image will be described with reference to FIG.

First, in step S125 of FIG. 3 or step 210 of FIG. 5, the service device 100 or the terminal device 200 extracts a reference eye and mouth from the user's static face image (S305). Here, the eye may be formed by first extracting the pupil, and then extracting the shape of the eye based on the pupil. The extraction of the eye and the mouth may be performed using general face recognition technology.

Next, the service apparatus 100 or the terminal apparatus 200 extracts the nose region by connecting the extracted eyes and mouths in an inverted triangle shape (S310), analyzes the edge and color change in the nose region, (S315).

In addition, the service device 100 or the terminal device 200 extracts the eyebrows by analyzing the color change in the area on the extracted eye (S320).

In addition, the service apparatus 100 or the terminal device 200 extracts feature points and contours of the face from the user's front face image (S325).

FIG. 7 is an exemplary view showing a state in which a feature portion of a face is extracted from a front face image of a user through the steps S305 to S325. In FIG. 7, white dots represent the feature points of each part of the face, and the respective face parts (for example, eyebrows, eyes, nose, mouth, contour, etc.) are extracted by connecting these feature points.

Finally, the service device 100 or the terminal device 200 combines the measured values and the statistical information of each part of the face on the basis of the minutiae of each part of the extracted face, (S330). As described above, the feature information of the face includes information necessary for polygon generation, such as the total length and width of the face, for each part of the face (eye, nose, mouth, eyebrow, outline, forehead, , The length of the chin, the length of the chin, the height of the nose, the length of the nose, the height of the mouth, the length of the mouth, the length of the eyes, and the height of the forehead. More specifically, the step (S330) may be performed by checking statistics appearing in the corresponding type based on the extracted face type information, and determining a possible value from the feature points of each part of the face extracted based on the statistic, The feature information of the face is calculated. For example, according to the statistic that the distance between the eyebrows and the line passing through the jaw line is within 60% of the total length of the face, the distance between the line traversing the eyebrow and the line passing the jaw line is measured in the front face image of the user , And this is applied to the statistic to derive the total length of the face. In the same way, feature information of the face such as nose height, nose length, mouth height, etc. is derived.

The face recognition inference method and the face recognition inference based service method using the same according to the present invention can be implemented in software form readable by various computer means and recorded in a computer readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a compact disk read only memory (CD-ROM), a digital video disk (DVD) Includes a hardware device that is specially configured to store and execute program instructions such as a magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), a flash memory, do. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be apparent to those skilled in the art. Furthermore, although specific terms are used in this specification and the drawings, they are used in a generic sense only to facilitate the description of the invention and to facilitate understanding of the invention, and are not intended to limit the scope of the invention.

According to the present invention, one or more of a user's facial image and a facial image can be inferred only from a facial image of a user photographed through a general camera or the like, thereby realizing a more realistic and stereoscopic user's image without expensive equipment, In addition, various application services based on face recognition can be provided based on the side face image and / or the rear face image deduced from the user's front face image.

In addition, according to the present invention, the side face image and the rear face image inferred from the user's front face image are applied to various virtual beauty experience services including hair and makeup, It is possible to provide an enhanced virtual beauty experience service, and the service provider has an excellent effect of reducing costs for implementing a more realistic and stereoscopic virtual beauty experience service.

100: face recognition based service apparatus 200: terminal apparatus
110: communication unit 120: face recognition unit 130: virtual beauty service unit
140: storage unit 210: communication unit 220: input unit
230: camera part 240: control part 250: output part
260:

Claims (13)

A communication unit for transmitting and receiving data to and from a terminal device via a network; And
A facial image of the user is extracted from the received frontal facial image, a facial feature information is compared and analyzed to extract facial type information, And a face recognition unit for generating a three-dimensional polygon for the front face based on the information and the type information of the face and inferring an image of a certain angle of the user's face from the generated three-dimensional polygon. The method according to claim 1,
And a virtual beauty service unit for generating a virtual beauty modeling image, which is expressed in a direction of a certain angle among the faces of the user, by combining the image of the user's face with a specific beauty object to provide the virtual beauty modeling image to the terminal device . [Claim 3 is abandoned upon payment of the registration fee.] 3. The method of claim 2,
A face type DB for storing information on a plurality of face types, which are compared with the face feature information in the face recognition unit and are divided into at least one of face type, race, gender, and age;
A facial image DB storing an image of a certain angle of a user's face deduced from facial feature information and face type information in the facial recognition unit; And
And a beauty object DB for storing a plurality of beauty objects to be used by the virtual beauty service unit. An input unit for generating a user input signal according to a user operation;
A camera unit for photographing a front face image of a user;
Extracts feature information of the face from the front face image, extracts the type information of the face by comparing and analyzing the face feature information, and then generates a three-dimensional polygon for the front face based on the feature information of the face and the type information of the face And a face recognition module for inferring an image of a certain angle of a face of the user from the face recognition module, wherein the control unit controls the camera unit according to an input signal of the user to acquire a front face image of a user, A controller for generating an image of a user's face at a predetermined angle from the front face image of the acquired user; And
And an output unit for outputting an image of a certain angle of the user's face under the control of the control unit. [Claim 5 is abandoned upon payment of registration fee.] 5. The apparatus of claim 4, wherein the control unit
Further comprising a virtual modeling module for generating a virtual beauty modeling image by combining an image of a predetermined angle among the faces of the user and a beauty object selected by the user. [Claim 6 is abandoned due to the registration fee.] 5. The method of claim 4,
The facial feature information and the face type information to be compared with the facial feature information are divided into at least one of a front face image, a face form, a race, a gender, and an age of a user, Further comprising a storage unit having at least one of an image of a certain angle of a face of the user and a plurality of beauty objects. [7] has been abandoned due to the registration fee. 6. The method of claim 5,
And a communication unit for transmitting and receiving data through the network,
Wherein data for inference or virtual beauty modeling of an image of a certain angle among the faces of the user is received from the service device. Obtaining a front face image of a user;
Extracting facial feature information from the frontal facial image;
Comparing the feature information of the extracted face with the pre-classified type to extract face type information; And
Generating a three-dimensional polygon for the front face by combining the extracted feature information of the face and the type information of the face, and generating an image of the user's face at a certain angle therefrom; Recognition inference method. [Claim 9 is abandoned upon payment of registration fee.] 9. The method according to claim 8, wherein the step of extracting feature information of the face
Extracting eyes and mouth from the frontal face image;
Extracting the nose region by connecting the extracted eyes and mouths in an inverted triangle shape;
Extracting detailed information of the nose by analyzing edge and color changes in the nose region;
Extracting an eyebrow by analyzing a color change in an area on the extracted eye;
Extracting feature points and contour lines of a face; And
And extracting facial feature information of a user necessary for polygon generation by combining measured values and statistical information of each extracted facial region. [Claim 10 is abandoned upon payment of the registration fee.] 9. The method according to claim 8, wherein the type information of the face
A face type, a race, a gender, and an age. [Claim 11 is abandoned upon payment of the registration fee.] 9. The method according to claim 8, wherein the feature information of the face
A face length, a length of a mouth, a length of an eye, a height of a forehead, a total length and a width of a face, a length of a chin, Inference method. [12] has been abandoned due to the registration fee. Obtaining a front face image of a user;
Extracting facial feature information from the frontal facial image;
Comparing the feature information of the extracted face with the pre-classified type to extract face type information;
Generating a three-dimensional polygon for the front face by combining the extracted feature information of the face and the type information of the face, and generating an image for a certain angle of the user's face;
Receiving selection information of a beauty object that the user wishes to experience;
And generating a virtual modeling image represented by a direction of an image of a certain angle of the user's face by combining the user's selected hair object with an image of a certain angle of the user's face. Recording medium.
delete

Images