JP5442966B2 - GAME DEVICE, GAME CONTROL METHOD, GAME CONTROL PROGRAM, AND RECORDING MEDIUM CONTAINING THE PROGRAM - Google Patents

Description

  The present invention relates to a game device, a game control method, a game control program, and a recording medium on which the program is recorded, and in particular, a game device and a game for easily simulating a makeup desired by a user on a face image The present invention relates to a control method, a game control program, and a recording medium on which the program is recorded.

  Conventionally, as one simulation technique, a simulation technique is known in which a user's face is photographed and a user's favorite makeup is synthesized from the photographed image.

In addition, when applying makeup to one's own face, since information on makeup methods and the like is generally presented in various magazine media, the makeup method desired by the user is selected with reference to the information. However, in reality, it is not only subjective but objective analysis of what type of face it is, and not just presenting makeup information that best suits the person, Therefore, there is a demand for a device that displays an image of what kind of face it is to be applied. For example, “(1) Extract each part of the face from the captured face image”, “ There are apparatuses that respectively implement methods such as (2) Divide by type ”and“ (3) Morph and paste together with a detected part of a preset makeup for each type ”(for example, (See Patent Documents 1 and 2.)
JP 2001-148018 A JP 2000-354517 A

  In the method disclosed in Patent Document 1 described above, a face image is captured from a camera, a face shape type classification operation is performed, and advice based on the face shape is given. It was only displayed, and as a result of following the advice, it was not possible to confirm how the face changed, and only the advice was known.

  The technique disclosed in Patent Document 2 described above forms an image of a face shape having a predetermined impression, and pastes the texture of a real human face on the face shape. However, in order to carry out this method, the only way to do this is to own or install equipment that is capable of carrying out the method. In other words, such devices are generally expensive and difficult to own, and if many users want to use such devices, the specialized location where the device is located, such as in a department store. It could not be carried out without going to the cosmetics corner, beauty salon, counter, etc. Therefore, going to a specialized place where such a device is located, in addition to the hassle of having to go out to the user, has to purchase some product instead of having the device used. Because it was expected that it would have to be, there were few opportunities for actual use.

  Such a situation makes it impossible for the user to confirm what makeup method suits his face, and knows the makeup method that suits his face based on objective data. It was unwilling to provide makeup without any provision, and even for a development company that develops cosmetics, it was impossible to provide a suitable makeup method based on objective data for the user, resulting in an unintended result.

  In addition, makeup is a daily activity for women, but when using a specialized device as shown in the prior art, the makeup method could not be confirmed on a daily basis. Therefore, there is a need for a program that can easily confirm the makeup method most suitable for the user himself provided based on objective data.

  In addition, when a makeup simulation is performed using a portable game device that is widely distributed in advance, a dedicated makeup simulation device installed in advance at a cosmetic corner or the like in a store is simulated. Since it does not have a dedicated CPU (Central Processing Unit) for image processing, for example, the display resolution (about 192 x 256 pixels), dot pitch (about 0.24 mm), about 260,000 color display In many cases, the performance of the imaging device used for such a device is not so high, and the accuracy of the face analysis deteriorates, and the accurate analysis is performed. I could not.

  The present invention has been made in view of the above-described problems, and is a game device, a game control method, a game control program, and the program for easily simulating a makeup desired by a user on a face image. An object of the present invention is to provide a recording medium on which is recorded.

  In order to solve the above-described problems, the present invention employs means for solving the problems having the following characteristics.

According to the first aspect of the present invention, a game program for displaying a virtual makeup image by synthesizing a preset object in response to a user operation on the acquired face image is executed. A game device capable of storing a plurality of pieces of makeup information in advance, an imaging unit for acquiring the face image, and the position and size of each part of the face with respect to the face image obtained by the imaging unit. Face image extraction means for extracting facial feature information, and specific makeup face information from a plurality of makeup face information stored in advance from the storage means according to the face feature information extracted by the face image extraction means. A makeup face generation unit that generates a makeup image that matches the face image based on the facial feature information obtained by the face image extraction unit, and the makeup face generation unit; Display means for displaying the obtained post-makeup image, wherein the face image extraction means is an edge extraction means for extracting edges for at least one or more preset parts in the face image; and the edge extraction means Extraction using the mask image generated from the skin color determination based on the face image obtained by decomposing the color space into hue, saturation, and brightness, respectively, to the image obtained by the above-described mask extraction And an outer periphery of the region of the part extracted by the mask extracting unit is positioned by a plurality of preset points, and at least one of the positioned points is displayed on the display unit together with an enlarged image of the part. , operating said moving at least one point on the basis of the movement instruction said for at least one point by the input means, to correspond to the site And having an adjusting means for adjusting the area of applying makeup virtually.

According to the first aspect of the present invention, it is possible to easily simulate the makeup desired by the user on the face image. Moreover, each part of the face can be extracted with high accuracy. Therefore, each part of the face can be extracted even if an image is taken using a non-high definition camera. In addition, makeup can be intentionally applied to a desired position of the face image in correspondence with the face image, and entertainment can be improved.

The invention described in claim 2 includes classification means for positioning to any one of a plurality of face classifications set in advance from the face feature information extracted by the face image extraction means, and the makeup face generation The means selects specific makeup face information from a plurality of pre-stored makeup face information according to the face feature information set for each face classification obtained by the classification means, and is obtained by the face image extraction means. A post-makeup image adapted to the face image is generated based on the face feature information.

According to the second aspect of the present invention, it is possible to classify the face image in correspondence with the face feature information and confirm the makeup face based on the result.

The invention described in claim 3 has corrected and corrected the position information for makeup included in the makeup face information according to the face feature information composed of the position and size of each part obtained by the face image extraction means. It has a control means for controlling to store the position information in the storage means.

According to the invention described in claim 3 , since the set information can be continuously used, usability can be improved.

The invention described in claim 4 is characterized in that the makeup face generating means includes an advice presenting means for generating advice in accordance with the displayed makeup in addition to the displayed post-makeup image. To do.

According to the fourth aspect of the present invention, makeup can be applied to the actual face corresponding to the post-makeup image.

The invention described in claim 5 provides a problem suitable for the face image, and determination means for determining the achievement level of the problem from information input to the face image by the user in response to the problem; And a determination display control means for displaying the determination result of the achievement level of the task determined by the determination means on the display means.

According to the fifth aspect of the present invention, knowledge of makeup can be increased and entertainment can be improved by responding to the problem with makeup.

The invention described in claim 6 executes a game program for displaying a virtual makeup image by synthesizing a preset object in response to a user operation on the acquired face image. A method for controlling a possible game apparatus, wherein an accumulation step for accumulating a plurality of makeup information in advance, an imaging step for acquiring the face image, and a position of each part of the face with respect to the face image obtained by the imaging step A face image extracting step for extracting facial feature information consisting of size and size, and a specific makeup from a plurality of makeup face information accumulated in advance by the accumulating step according to the face feature information extracted by the face image extracting step A makeup face generation step of selecting face information and generating a makeup image adapted to the face image based on the face feature information obtained by the face image extraction step of the makeup face information; and the makeup face A display step of displaying on the display means for displaying the post-makeup image obtained by the composition step, wherein the face image extraction step is an edge for extracting an edge for at least one or more parts set in advance in the face image By combining the image obtained by the extraction step and the edge extraction step using a mask image generated from skin color determination based on the face image obtained by decomposing the color space into hue, saturation, and brightness, respectively. , A mask extracting step for extracting the part, and an outer periphery of the region of the part extracted by the mask extracting step is positioned by a plurality of preset points, and at least one of the positioned points is enlarged. displayed on the display means together with the image, based on the movement instruction at least one point for said at least one point by the operation input unit So moved, and having an adjusting step of adjusting the area where virtually performing makeup corresponding to the site.

According to the sixth aspect of the present invention, it is possible to easily simulate the makeup desired by the user on the face image. Moreover, each part of the face can be extracted with high accuracy. Therefore, each part of the face can be extracted even if an image is taken using a non-high definition camera. In addition, makeup can be intentionally applied to a desired position of the face image in correspondence with the face image, and entertainment can be improved.

According to the seventh aspect of the present invention, there is provided a game in which a display object is displayed on an acquired face image by synthesizing a preset object in response to a user operation and applying a virtual makeup. A computer constituting a game device capable of executing a program is configured to store in advance a plurality of makeup information, an imaging unit that acquires the face image, a position of each part of the face with respect to the face image obtained by the imaging unit Face image extraction means for extracting face feature information consisting of size and size, and in accordance with the face feature information extracted by the face image extraction means, a plurality of makeup face information stored in advance from the storage means Makeup face generation means for selecting makeup face information and generating a post-makeup image adapted to the face image based on the face feature information obtained by the face image extraction means. The face image extracting means extracts an edge for at least one or more predetermined parts in the face image, and extracts a color space from the image obtained by the edge extracting means. , By using a mask image generated from the skin color determination based on the face image decomposed into the saturation and the brightness, respectively, and extracting the part, and the mask extraction means extracted by the mask extraction means The outer periphery of the region of the region is positioned by a plurality of preset points, at least one of the positioned points is displayed on the display unit together with the enlarged image of the region, and the movement with respect to the at least one point is performed by the operation input unit and wherein moving at least one point on the basis of the instruction, to adjust the area virtually performing makeup corresponding to the site And having a settling unit.

According to the seventh aspect of the present invention, the makeup desired by the user can be easily simulated for the face image by the game control program. Moreover, each part of the face can be extracted with high accuracy. Therefore, each part of the face can be extracted even if an image is taken using a non-high definition camera. In addition, makeup can be intentionally applied to a desired position of the face image in correspondence with the face image, and entertainment can be improved.

The invention described in claim 8 is a computer-readable recording medium in which the game control program according to claim 7 is recorded.

According to invention of Claim 8 , a game control program can be installed easily using a recording medium. Moreover, each part of the face can be extracted with high accuracy. Therefore, each part of the face can be extracted even if an image is taken using a non-high definition camera. In addition, makeup can be intentionally applied to a desired position of the face image in correspondence with the face image, and entertainment can be improved.

  According to the present invention, in a game device, a game control method, a game control program, and a recording medium on which the program is recorded, makeup for a user can be easily simulated.

<Outline of the present invention>
The present invention, for example, is a dedicated image processing that performs processing of an image acquired by an imaging means that is widely distributed without using a specialized device such as a counter installed in a cosmetics corner of a department store or the like. Provided is game software content in which a user can easily analyze his / her face using a portable terminal (game device or the like) that does not have a means, and can confirm a makeup face based on the result.

  Specifically, a face image is captured, the captured face image is analyzed, a makeup face suitable for the person is selected from the analysis result, and the selected makeup face is formed into a desired form for the person's face. By displaying after correction, a makeup face or the like that best suits the person is presented based on objective information.

  Hereinafter, preferred embodiments of a game device, a game control method, a game control program, and a recording medium storing the program according to the present invention will be described with reference to the drawings.

<Simulation game device: schematic configuration example>
FIG. 1 is a perspective view of a game machine showing a first embodiment of the present invention. In FIG. 1, a game machine (game device) 10 shown as an example includes a substantially rectangular first display screen 12 and a second display screen 14 configured by a liquid crystal display, and an image processing device in a housing 16. Is housed. The first display screen 12 and the second display screen 14 can be used in a horizontally long state as shown in FIG. 1, or can be used in a vertically long state.

  The first display screen 12 for displaying the game screen is supported by the upper housing 16a, and the second display screen 14 for displaying the game screen is similarly supported by the lower housing 16b, and the upper housing 16a. The lower housing 16b is rotatably connected via a hinge. The screens displayed on the first display screen 12 and the second display screen 14 are not limited to moving images or still images. When a moving image is displayed on the first display screen 12, An image obtained by capturing the image displayed on the first display screen 12 can be displayed on the display screen 14 as a still image, or vice versa. The displayed image can be a virtual space image composed of polygons or the like by a display means, or a real space image captured by a camera to be described later.

  Also, a speaker sound release hole 18 is formed on the left end side of the upper housing 16a, and a direction indicating switch (cross switch) 20a, a start switch 20b, and a select switch 20c are provided as operation switches 20 on the lower housing 16b. An operation switch (A button) 20d, an operation switch (B button) 20e, an operation switch 20f, an operation switch 20g, an operation switch (left push button) 20L, and an operation switch (right push button) 20R are arranged.

  Here, the direction indicating switch 20a functions as a digital joystick, and by operating one of the operation units, the direction of the cursor that can be operated by the user (player) is specified, or a desired makeup simulation is realized. It is possible to instruct the selection of an item to be performed, to instruct the moving direction of the object presented by the software, and to instruct the moving direction of the cursor. The shape of the button is easy to input in four directions, but it is also possible to input diagonally. The start switch 20b includes a push button, and is used to start (resume) or temporarily stop a game (simulation or the like). The select switch 20c is composed of a push button and is used for selecting a game mode.

  The operation switch 20d is configured as a push button as the A button, and can perform an operation other than a direction instruction, for example, an operation such as determination of an item selected by the user, execution of a simulation, and the like. The operation switch 20e is a push button as the B button, and is used for changing the game mode selected by the select switch 20c or canceling the action determined by the A button 20d.

  Furthermore, the operation switch (left push button) 20L and the operation switch (right push button) 20R are configured by push buttons, and the operation switches 20L and 20R are used for auxiliary operations of the A button 20d and the B button 20e. Can do.

  These switches can have different roles as described above, or they can have different roles from those described above. Can be changed as appropriate.

  On the second display screen 14, a touch panel 22 is attached as operation input means. As this touch panel 22, for example, any one of a resistive film type, an optical type (infrared type), a capacitive coupling type, an ultrasonic type, and an electromagnetic induction type can be used. The touch panel 22 detects the coordinate position of the stick 24 or the like and outputs it as coordinate position data when an operation of pressing or touching the top surface with a stick or stylus pen (pen) 24 or a finger is performed. It is supposed to be. The stick 24 can be stored in the storage portion 26 of the lower housing 16b, and is taken out from the storage portion 26 as necessary.

  Further, as shown in FIG. 1, the game machine 10 includes an upper housing 16a and an imaging unit (camera with a lens) 28 for photographing a face or the like of a target person who performs a makeup simulation of a user (player) or the like. Connected by a connector built in the game machine 10.

  As shown in FIG. 1, a memory card (or game card) 30 as a recording medium is detachably attached to a connector built in the game machine 10. For example, the memory card 30 is inserted through an insertion port 32 formed on the back side of the lower housing 16b and is attached to the connector. When the memory card 30 is attached to the connector, the memory card 30 can be accessed from the CPU core of the game machine 10.

<Game machine 10: block diagram>
Next, a block configuration example of the above-described game machine 10 will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a block configuration in the game machine. As shown in FIG. 2, the game machine 10 houses a circuit board 40 for mounting an image processing apparatus.

  On the circuit board 40, a CPU core 42, a bus 44, a first connector 46, a second connector 48, and the like are mounted. The CPU core 42 is connected to the first second connector 48 and the second connector 48 via the bus 44, a RAM (Random Access Memory) 50, and a first graphics processing unit (GPU: Graphics Processing). (Unit) 52, a second GPU 54, an input / output interface circuit (hereinafter referred to as I / F circuit) 56, and a controller 58.

  The camera 28 attached to the first connector 46 includes a lens 28a and the like, and for example, captures the whole face image or each part of the face image of the subject who performs the makeup simulation of the user or the like with a predetermined size. That is, the lens 28a has a function of enlarging or reducing, and for example, the image of each part such as the entire user's face or eyes and mouth is displayed on the first display screen 12 or the second display screen 14 with a predetermined size. May be displayed.

  The memory card 30 attached to the second connector 48 includes a ROM (Read Only Memory) 30a and a RAM 30b, both of which are connected to each other via a bus. For this reason, the CPU core 42 can access the ROM 30 a and the RAM 30 b of the memory card 30 via the bus 44 and the second connector 48.

  The ROM 30a stores a game program for a game (virtual game) to be executed on the game machine 10, image (character image, background image, item image, message image, etc.) data, and sound (music) data necessary for the game. (Sound data) and the like are stored in advance. Further, the RAM 30b relates to game in-progress data and game result data, for example, information automatically acquired at an event, makeup simulation result, makeup method, etc. when the acquired item type or story is advanced more than a certain level. Various data are stored.

  The RAM 50 is used as a buffer memory or a working memory. For example, the CPU core 42 loads a game program, image data, sound data, and the like stored in the ROM 30a of the memory card 30 into the RAM 50, and performs control to execute the loaded game program. In this case, the CPU core 42 performs control to execute game processing while storing data (game data and flag data) temporarily generated in accordance with the progress of the game in the RAM 50.

  Each of the first GPU 52 and the second GPU 54 constitutes an element of the drawing unit, and is composed of, for example, a single ASIC, receives a graphic command from the CPU core 42, and receives a game image ( (Game screen) data (screen data) is generated. In this case, the CPU core 42 provides an image generation program (a program included in the game program) necessary for generating screen data to the first GPU 52 and the second GPU 54 in addition to the graphic command. .

  A first video RAM (hereinafter referred to as VRAM) 60 is connected to the first GPU 52, and a second VRAM 62 is connected to the second GPU 54. The first GPU 52 draws the created screen data in the first VRAM 60, and the second GPU 54 draws the created screen data in the second VRAM 62. The GPUs 52 and 54 and the VRAMs 60 and 62 are connected to the controller 58, respectively.

  The controller 58 includes a register 64 composed of, for example, 1 bit, and stores a value of “0” or “1” in accordance with an instruction from the CPU core 42. The controller 58 outputs the screen data drawn in the first VRAM 60 to the first display screen 12 when the data value of the register 64 is “0”, and when the data value of the register 64 is “1”. The screen data drawn in the second VRAM 62 is output to the second display screen 14.

  The operation switch 20, the touch panel 22 and the speaker 66 are connected to the I / F circuit 56, and signals from the operation switch 20 and the touch panel 22 are input to the CPU core 42 via the I / F circuit 56, and the CPU core. 42 is output to the speaker 66 via the I / F circuit 56. For example, when one of the operation switches 20 is operated, an operation signal (operation data) of the operated switch is input to the CPU core 42 via the I / F circuit 56. Also, coordinate position data from the touch panel 22 is input to the CPU core 42 via the I / F circuit 56. Furthermore, the CPU core 42 reads out sound data necessary for the game, such as an announcement for performing a makeup simulation, game music, sound effects, or a sound of a game character (imitation sound) from the RAM 50, and via the I / F circuit 56. Output from the speaker 66.

<Configuration example of RAM memory map>
Here, a configuration example of the memory map of the RAM 50 described above will be described with reference to the drawings. FIG. 3 is a diagram illustrating a configuration example of a memory map of the RAM according to the present embodiment.

  The RAM 50 includes a program storage area 70 and a data storage area 72 as shown in FIG. Here, in the program storage area 70, a game surface processing program 80, a controller input detection program 82, a touch panel input detection program 84, an image generation program 86, a main image display program 88, a sub screen display program 90, a player character display control program. 92, an object display control program 94, and the like are stored. In the data storage area 72, image data 100 and sound data 102 are stored, and an object data buffer area 104, an operation data buffer area 106, and a coordinate position data buffer area 108 are provided.

  The game main process program 80 is a program for executing a game main process of a virtual game such as a makeup simulation. The controller input detection program 82 is a program for detecting operation data input from the controller, that is, the operation switch 20, and writing the detected operation data in the data storage area 72. The touch panel input detection program 84 is a program for detecting coordinate position data input from the touch panel 22 and writing the detected coordinate position data in the data storage area 72.

  The image generation program 86 is a program for generating a game image. The image generation program 86 is given to each of the first GPU 52 and the second GPU 54 by the CPU core 42, and the first GPU 52 and the second GPU 54 respectively convert the screen data according to the image generation program 86. The first VRAM 60 and the second VRAM 62 are generated.

  The main screen display program 88 is a program for giving a display command to the controller 58 when a game screen is displayed on the first display screen 12 as a main screen. The sub screen display program 90 is a program for giving a display command to the controller 58 when a capture screen is displayed as a sub screen on the second display screen 14.

  The player character display control program 92 is a program for controlling the display of the player character on the game screen in accordance with a user operation. The object display control program 94 is a program for controlling the display of objects on the game screen. Here, the object is a point of each part extracted for analyzing a face image to be described later, various items displayed on the screen for allowing the user to select the operation content, navigation for performing a makeup simulation Meaning a character, a player character, or an object that gives advice on a makeup method, an announcement, a makeup method, and the like, for example, a person object, an item object, and background objects. Note that the advice of the makeup method is not limited to text information, but may be audio information, still image, video information including a moving image, or a combination of two or more thereof.

  The image data 100 stored in the data storage area 72 is data necessary for generating an image of an object, a player character, and the like, and is icon image data, message image data, and the like. The sound data 102 is data for outputting sounds (music) necessary for the game.

  The object data buffer area 104 is an area for temporarily storing data related to an object displayed on the game screen when a user performs a capture operation. The operation data buffer area 106 is an area for temporarily storing operation data detected by the CPU core 42 in accordance with the controller input detection program 82. The coordinate position data buffer area 108 is an area for temporarily storing the coordinate position data detected by the CPU core 42 in accordance with the touch panel input detection program 84.

<Example of software>
The content of the game software content according to the present embodiment will be specifically described with reference to the drawings. The contents shown below are stored in a recording medium or the RAM 50 as a computer program.

  In this embodiment, face information for analyzing a face is acquired from a face image of a makeup simulation target person photographed by an imaging means such as a camera, and, for example, “recommended makeup” is automatically selected based on the acquired face information. Then, the selected makeup is generated and displayed on the face image, or the makeup subject is presented to the face of the subject who performs the makeup simulation in the “free makeup mode”, and the achievement level of the task is scored. It is possible to provide content (applications) and the like such as competing.

<Face image extraction process>
In the present embodiment, first, a face image extraction process for acquiring a face image that is a basis for performing a makeup simulation will be described with reference to the drawings.

  FIG. 4 is a flowchart showing an example of face image extraction processing in the present embodiment. The flowchart shown in FIG. 4 shows a method for acquiring a face image with high accuracy even in a game apparatus having an operation environment with relatively low hardware.

  In the face image extraction process shown in FIG. 4, first, the entire face image is picked up by the camera which is the above-described image pickup means (S01), the picked up face image is stored in a memory or the like (S02), and then stored. Noise removal is performed by filtering the face image with a time axis filter, a three-dimensional filter, or the like (S03).

  Next, face recognition processing is performed on the image from which noise has been removed by the processing of S03 (S04). Specifically, in order to detect the point coordinates for each part of the face included in the face image, store them in the buffer, and further smooth the image with respect to the time axis, for example, a constant such as 3 × 3 The pixels in the area are sorted by density or the like, and a median filter that is image processing using the value of the “center” pixel as the center value of the area is passed. For example, edge extraction processing, mask extraction processing, or the like is performed on points that are important for makeup such as the upper face line or the lower eye line of the face.

  Next, each part of the face is cut out (S05). Specifically, for example, face parts are cut out using HSV decomposition processing. The HSV is a color space composed of three components of hue (Hue), saturation (Saturation), and brightness (Value). The eye, lips, etc. included in an image photographed by separating these components, etc. Cut out for each part. Note that the saturation accuracy may be further increased by performing the modified HSV process.

  Next, a mat key is generated and chroma key composition processing is performed (S06), and the extraction points of each part obtained after the composition are stored in the storage means (RAM) or the like (S07). Further, the position and size of each part are stored with the peripheral part of the extraction point as the part itself (S08), and the point is displayed together with the face image (S09). Here, the displayed image is confirmed and it is determined whether or not correction is necessary (S10). If correction is necessary (YES in S10), the process returns to S01, and the face image is taken again. Perform subsequent processing. If no correction is necessary (NO in S10), the face image extraction process is terminated. Thereby, the position and size of each part of the face can be extracted from the face image obtained by the camera.

  Next, each process of S03-S06 mentioned above is demonstrated concretely.

<S03: Noise removal>
Here, the above-described noise removal will be described. The camera used in the present embodiment uses an imaging device or the like that is used for a device that does not have high performance such as resolution. Too much sticking is a problem. Therefore, in the present embodiment, the time axis filter is performed using the images of the previous frame and the current frame regarding noise, or the noise is removed using a three-dimensional filter.

  For example, in the case of time axis filtering, the above-described filtering is performed by replacing the pixel with the average value of other pixels based on the relationship between pixels at the same position between adjacent frames. In spatial filtering, based on the relationship between adjacent pixels in the same frame, the pixel is replaced with an average value of other pixels. Note that noise may be removed using other filtering processing.

<S04: Face recognition processing>
Next, the face recognition process (S04) described above will be described. In the face recognition process, for example, FSE (Face Sensing Engine (registered trademark)) manufactured by Oki Electric Co., Ltd. can be used. Specifically, as face recognition, for example, as disclosed in Japanese Patent Laid-Open No. 2000-339476, a face center line is obtained from image data obtained by photographing a user, and a face area is determined based on the center line. The face area obtained by the determination is divided into a left eye area, a right eye area, and a nose area, and the positions of the eyes are detected from image data in the left eye area, the right eye area, and the nose area, and the left eye area, the right eye area, and the nose area are detected. The face position is detected from the image data in the region.

  Note that normal FSE processing focuses on real-time detection, and is not suitable for strict feature point extraction processing for each part like this time. Therefore, in the present embodiment, edge extraction is performed on the upper face line, lips, eyebrows, eye corners, eye lower line, etc., and the coordinates of point detection are accumulated in a buffer and passed through a median filter with respect to the time axis, Further, the detection point is set so that the user can move at least one of the point points by himself / herself.

  Here, FIG. 5 is a diagram illustrating an example of edge extraction in the present embodiment. As shown in FIG. 5, for example, an upper face line (FIG. 5 (b)) of the face is extracted as an edge extraction with respect to the original image (FIG. 5 (a)) of the entire face of the user taken by a camera or the like. Lines (FIG. 5 (c)), lower lines of the left and right eyes (FIGS. 5 (d) and (e)), etc. are performed to detect each part. In addition, the site | part extracted is not limited to this in this invention, For example, you may extract a site | part of eyebrows or a nose. In addition, by performing extraction processing using a mask image on an image extracted by edge extraction, the extraction accuracy of each part can be improved even if the image is not a high-definition image.

<S05: Cutting out each part of face>
Next, the clipping process (S05) of each part of the face described above will be described. In the process of S05, a mask image is generated as the extraction of each part of the face. That is, the skin color is determined from the above-described HSV color space in order to cut out a face for each part from the input image. Here, FIG. 6 is a diagram illustrating an example of an image in HSV decomposition. As shown in FIG. 6, HSV decomposition is for decomposing an input image into hue (Hue), saturation (Saturation), and lightness (Value) as shown in FIG. 6 (a). Since HSV decomposition can ignore the brightness of an image, skin color recognition that is not affected by various environments becomes possible.

  However, in the HSV decomposition, since a color close to black, which should have low saturation, shows high saturation, noise may occur in the skin color region. In order to cope with this, a modified HSV process is performed as shown in FIG. Note that this modified HSV process differs depending on the performance of the camera that shoots the user, and therefore the modified HSV process can be omitted as necessary.

<S06: Composition Processing>
Next, the above-described combining process (S06) will be described. In the process of S06, a mat key (mask signal) is automatically generated using the face extraction points by the above-mentioned FSE, and the extraction accuracy is increased by chroma key composition. Here, FIG. 7 is a diagram illustrating an example of a mat key. Each part can be extracted with high accuracy by generating a mat key for a face line as shown in FIG. 7A and a mat key for a lip as shown in FIG. 7B. In addition to this part, parts such as eyes, nose and ears can be used as mat keys, so that highly accurate part extraction can be realized.

<Screen transition example of face image extraction in game software>
Next, a screen transition example of the face image extraction process described above will be described with reference to the drawings. FIG. 8 is a diagram illustrating an example of game screen transition of the game software in the present embodiment. FIG. 8 shows a screen for performing user registration in advance when using the game software in the present embodiment. 8A to 8I show examples in which the screens displayed on the display units (the first display screen 12 and the second display screen 14) of the game machine 10 are displayed on the left and right. ing. In addition, on one screen (left side of the screen), instructions for the user (navigation, announcement) are displayed. On the other screen (right side of the screen), the user inputs instructions such as a touch pen based on the contents of the instructions. An example in which instruction contents are input using a device or the like is shown. In addition, the announcements and the like shown below are not only displayed on the screen but also can be output as audio.

  First, in the shooting mode as shown in FIG. 8A, as shown on the left side of FIG. 8A, a CG character (such as a counselor) such as an avatar is “ When you are ready, start shooting. ”As shown in the right screen of Fig. 8 (a), precautions at the previous stage of shooting (for example, pin and , Etc.) is displayed.

  Next, as shown in FIGS. 8B and 8C, the brightness is selected by the exposure compensation function of the camera. Specifically, as shown in the left screen of FIG. 8B, the character displays a message such as “The camera has been activated. Adjust the brightness with the ○ button.” On the other screen, It is displayed as real-time video actually obtained from the camera. Here, the user adjusts the brightness by using the ○ button for designating the brightness, and acquires an image having a certain predetermined brightness as shown in FIG. At this time, if the game apparatus 10 has a function of detecting brightness, the brightness may be adjusted according to the detected value.

  Next, as shown on the right side of FIG. 8 (d), when the user's face enters a preset area on the screen, the countdown is automatically performed from a predetermined number (FIG. 8 (d) right side). (Above), make an announcement to take a picture. At this time, as shown on the left side of FIG. 8D, the character displays a message such as “Shoot. Please wait ...” and take a picture of the face image of the user (user). I do.

  Next, with respect to the captured image, the FSE process described above extracts points on each part (each part) of the entire face surface, eyebrows, eyes, nose, and mouth, and the contents are shown on the right side of FIG. Is displayed. Note that a predetermined number of different colors are displayed for the points of each part, but in the present invention, the color of each point, the type of each part, and the like are not particularly limited. Note that the right screen of FIG. 8E is provided with a re-button for re-shooting. By selecting this button, the process returns to the shooting stage of FIG. 8A and the subsequent processing is continued. Can be done.

  Next, the point of each part is adjusted. Specifically, as shown in FIG. 8 (f), a message “Please adjust the reference point of the mouth and nose” is displayed, and the image of the mouth is enlarged and displayed together with the extraction point. By touching (selecting) and moving at least one point to be adjusted from a plurality of points on the outer periphery, each point can be moved to a predetermined position. At this time, each point can be moved so as to be traced with the touch pen, or can be adjusted with the cross key after selecting at least one of the points. Moreover, the point to be moved can be blinked, and the user can easily be notified of which point is moved to the target.

  Next, as shown in FIG. 8G, a message “Please check your mouth and nose points” is displayed, and you can touch and move the point you want to adjust. At this time, fine adjustment can be performed using the cross key.

  Further, as shown in FIG. 8H, for example, a message “Please adjust the left eye and eyebrow reference points” is displayed, and the eye can be moved by touching the point to be adjusted. . At this time, the point to be adjusted can be finely adjusted with the cross key.

  Further, after the left eye and eyebrow reference points are determined, a message “Please check the left eye and eyebrow points” is displayed as shown in FIG. It can be moved by touching.

  At this time, fine adjustment can also be performed with a cross key provided in the game apparatus 10. By applying the above-described processing to the right eye and face line as well, it is possible to clearly set the point for makeup.

  In addition, by performing readjustment of points as described above, the position and size of each part can be extracted with higher accuracy. In addition, the makeup simulation can be performed by intentionally shifting the position and size of each part from the actual position, and entertainment can be improved.

  Therefore, based on the content extracted by the face image extraction means described above, it can be used for content as shown below. Moreover, the makeup desired by the user can be easily simulated for the face image using the extraction result of the face image extraction process.

  The face image extraction process described above may be performed in advance as a user registration process before each content of the game using the face image extraction process described later.

<Content example>
Next, an example of game content using the face image extraction process described above will be described with reference to the drawings. As an example of game content using the face image extraction process described above, in the following description, each content such as “recommended makeup”, “task mode”, “free makeup”, etc. will be described. The content is not limited to this.

<Recommended makeup>
First, the recommended makeup will be explained. The recommended makeup is for generating and displaying the makeup selected by the user on the face image. Specifically, after explaining the precautions when photographing a face, a face image is captured from the face photographing by the facial scanning, and the facial parts (each part) are extracted by the face recognition engine. In addition, the optimum makeup method desired by the user is presented from the extracted results. Further, in the extraction of facial parts, readjustment by the user can be performed, and a facial feature classification method positioned in any one of a plurality of facial classifications set in advance from the extracted facial feature information. It can be carried out.

<Recommended makeup: Overall flow>
FIG. 9 is a diagram showing an example of an overall processing flowchart for recommended makeup. In the process shown in FIG. 9, first, an image of the user is acquired, the above-described point extraction (face image extraction process) is performed on the acquired image (S11), and then, from the extracted feature point information of each part. A face analysis is performed based on a predetermined face classification (S12). Next, a mask for makeup simulation is determined from the face analysis result shown in S12 (S13), and virtual makeup processing is performed by applying makeup that matches the face image from the determined mask (S14). Further, a makeup explanation process is performed for presenting the post-makeup image obtained from the processing result of S14 together with the makeup explanation of each part in the recommended makeup (S15).

<S12: About facial analysis processing>
Next, the face analysis process (S12) described above will be specifically described. With regard to the facial feature analysis processing shown in S12, for example, facial feature classification can be performed on a facial image photographed using a facial feature classification method such as that disclosed in Japanese Patent No. 3828285.

  Specifically, two-dimensional coordinates are set on the plane, “the length of the face and the arrangement state of each part of the face (eyebrow, eye, nose, mouth)” on the vertical axis, and “each part of the face ( Shape of eyebrows, eyes, nose, mouth) "is set as the coordinate axis. The vertical axis indicates, for example, that the face length is “short” to “long”, each part of the face is “distant from the left / right center” to “close to the left / right center”, or “the part is This indicates a change from "closed" to "distant from the vertical center". This is consistent with changes from child-to-adult growth, and can be seen as indicating a change in balance during child-to-adult growth, showing the axis defined as the balance axis. . The horizontal axis represents, for example, the shape characteristic of each part of the face being curved or linear, and shows an axis defined as a form axis.

  In other words, in the form and impression located in each quadrant of the coordinate space, for example, if the face is long and each part of the face is curved, the gentle feminine impression is strengthened, or the face is short and the shape is linear The relationship between form and impression is clarified, such as giving a fresh and lively impression.

  In this way, by classifying faces by using the length of the face and the arrangement state of each part (part) of the face of the eyes, eyebrows, mouth, and nose, and the shape of each part of the face as an index, the face is classified according to features from the form. It is possible to classify and classify according to the evidence received from the shape of the face. And by classifying in this way, it is possible to produce an image accurately when performing make-up, for example. Further, by assigning a classification to a customer list such as a beauty salon, it can be used as a guideline for creating an image suitable for the user.

  In the present embodiment, face determination is performed based on the above-described face map classification method. For example, an intersection point between a straight line passing through the pupil and the hairline and an intersection point between the straight line passing through the left and right ends of the mouth and the face outline are extracted as feature points from the user's image. Then, from the coordinates of each feature point, the angle formed by the feature points of the eyes and chin, the aspect ratio of the face, and the width ratio of the eyebrows and the face are calculated. Also, the chin width and the face length ratio, the chin angle formed by the facial contour feature points, the eye-mouth distance and the face length ratio are calculated. Each type such as a child type, an adult type, a curve type, and a straight line type is determined by comparing the calculated result with a preset reference value. The center of each axis can be determined as the average face golden balance (standard type).

  The above-mentioned reference value uses an average face (average face) value created by morphing (transforming an image) with a computer from the faces of many Japanese women. Based on this judgment result, the user's face type is adult type, child type, straight line type, curve type, child / straight type, child curve type, adult / straight type, adult / curve type, average face type, and so on. It can be classified into 15 types on 4 axes (arrows) and displayed on the display. In addition, the type for analyzing a user's face is not limited to the above-mentioned type, and can be classified into other face types. Moreover, in the above method, the face is limited to the Japanese face, but not limited to the Japanese face, any person in the world creates a specific face type, It is also possible to classify the type based on it.

<S14: Virtual makeup processing>
Next, the virtual makeup process (S14) described above will be specifically described with reference to a flowchart. FIG. 10 is a flowchart illustrating an example of the virtual make process. In the virtual makeup process shown in FIG. 10, first, the face-captured image is expanded to a predetermined bitmap (for example, a 32-bit map) (S21), and then a makeup process for the base, lips, etc. is performed (S22). Specifically, blend processing is performed using a mask image or the like. Next, processing for eyebrows, cheeks, mascaras, eye shadows, etc. is performed (S23). Specifically, the reference mesh processing is performed on the face image, and the resulting mesh is modified according to the points stored in the face image extraction processing described above to generate a deformed mesh, and each part such as eyebrows and lips A color is set for the texture generated in advance and the texture is input. Next, the texture of each part obtained by the process of S23 is synthesized with the face image by blending by texture mapping (S24), and the synthesized image is displayed on the display screen (S25). Further, the image displayed on the screen by the process of S25 is recorded (S26).

<Screen transition example>
Next, a screen transition example of the recommended makeup process described above will be described with reference to the drawings. FIG. 11 is a diagram illustrating an example of screen transition in the recommended makeup process. Note that FIG. 11A shows the screen transition of the previous stage of processing, and FIG. 11B shows the screen transition of the subsequent stage following (A) in FIG. 11A. In addition, the face image displayed on the screen in FIG. 11 is actually a color face image acquired in advance by photographing or the like.

  In the diagram shown in FIG. 11, after the user's face data is registered, the face type analysis is performed. That is, a predetermined one is classified from the five classifications according to the face makeup set in advance. As shown in FIG. 11, when each content is started, the character explains the “face type analysis” as shown on the left side of the screen. Which type the person's face belongs to is displayed on the display screen. Moreover, the adult face degree and the curve face degree can be digitized according to the result, and the digitized result can be displayed. Moreover, the results such as the analyzed screens and numerical values can be accumulated as they are. Further, based on this result, the recommended makeup display as shown in FIG. 11 is performed corresponding to the face.

  In addition, as shown in FIG. 11, when the recommended makeup is selected, the face type (in this embodiment, for example, five types (“friendly / feminine”, “cute / cute”, “fresh / active”, “cool / sharp” ”,“ Standard ”)) Set each recommended makeup. In addition, comments to be presented on the display screen or the like of the game apparatus 10 for each face type or various information for performing makeup are set in advance.

  FIG. 12 is a diagram showing an example of various information used in the present embodiment. FIG. 12A shows an example of announcement contents by type, FIG. 12B shows an example of makeup type information for each part, and FIG. 12C shows an example of makeup color information for each part. Note that the various types of information shown in FIG. 12 are stored in advance in storage means such as the RAM 50, and can be read out, added, modified, or deleted as necessary.

  The comment content list shown in FIG. 12A includes, for example, each part (part) (for example, “foundation”, “brow”, “eye shadow”, “eye line”, “mascara”, “ Lipstick "," blush "etc.) and advice information for making recommended makeup for each part of each type (for example, how to spread foundation, how to make it look easy with eyeline, lipstick In order to present a cute display method using characters, sounds, images, videos, etc., link information to the dictionary stored in the storage means or the like is stored. Accordingly, appropriate information can be quickly presented to each part for each type selected by the user or selected in the recommended makeup process.

  In the type information shown in FIG. 12B, one or a plurality of types are accumulated for each part (part) of each type (face map type), and one of them is selected and accumulated in the item of “folder”. The corresponding type information is displayed on the screen. As for the color information shown in FIG. 12C, one or a plurality of types are accumulated for each part (part) of each type (face map type). Display the corresponding color information stored on the screen. In this embodiment, since a plurality of colors can be used, for example, in the case of eye shadow, three patterns of color information are accumulated in a “folder” and displayed on the screen as needed. be able to. Here, the various types of information illustrated in FIGS. 12A to 12C may be managed with different IDs, and may be associated with each other as necessary.

  Based on the content described above, different content for each type can be presented to the user using the character. Further, the generated simulation image can be stored in the storage means according to the content of the user instruction. Each item is not limited to the seven items described above, and the number may be reduced or increased.

<Makeup by situation>
Next, the makeup by situation included in the recommended makeup described above will be described in detail. FIG. 13 is a diagram illustrating an example of screen transition in situation-specific makeup. Note that FIG. 13A shows the screen transition of the previous stage of processing, and FIG. 13B shows the screen transition of the subsequent stage following (B) in FIG. 13A. In addition, the face image displayed on the screen in FIG. 13 is actually a color face image acquired in advance by photographing or the like.

  As shown in FIG. 13, on one screen (for example, the left screen) of the game apparatus 10, the user selects face data that the user wants to make from a pre-captured face image, and is set in advance to the selected face. For the purpose of face type and makeup, for example, you can select one from items such as business, dating, friends, forum, etc. Also, business (usually), business (conference / presentation), shopping, outdoor, wedding, funeral, Select one from the contents of the party. Further, by selecting a classification such as warm color, cold color, or monotone, it is possible to display post-makeup images by situation. The post-makeup image can be stored in storage means such as a RAM.

<Trend makeup>
Next, the trend makeup included in the recommended makeup described above will be specifically described. FIG. 14 is a diagram illustrating an example of screen transition in trend makeup. Note that the face image displayed on the screen in FIG. 14 is actually a color face image acquired in advance by photographing or the like.

  As shown in FIG. 14, on one screen (for example, the left screen) of the game machine 10, the desired makeup result is displayed on the screen by selecting the face type of the user's desired impression. For example, you can select the target image for makeup from the images that have been taken so far, and select the season (spring, summer, autumn and winter) and the trend (elegant, clean, natural, cool), etc. The rear image can be displayed. The post-makeup image can be stored in storage means such as a RAM.

<S15: About makeup commentary>
Next, the makeup explanation process (S15) described above will be specifically described. FIG. 15 is a flowchart illustrating an example of makeup commentary processing. In the makeup commentary process shown in FIG. 15, it is first determined whether or not a part to be makeup is selected (S31). If a part to be makeup is not selected (NO in S31), the process is continued until it is selected. If no input is made for a predetermined time, the makeup commentary process may be terminated. If the part to be made is selected (YES in S31), the corresponding part is enlarged from the stored contents stored in advance in the storage means corresponding to the part and displayed on the display screen (S32). At this time, the explanation about makeup is also displayed.

  Here, it is determined whether or not the makeup commentary mode is to be ended (S33). If the makeup commentary mode is not to be ended (NO in S33), the processing returns to S31 and the subsequent processing is performed. If makeup explanation mode is to be ended (YES in S33), makeup explanation mode is terminated.

<Problem mode processing>
Next, as a content example in the present embodiment, processing contents in the assignment mode will be described with reference to the drawings. In the task mode, a makeup method that suits the user is presented to the user as a task, and the user inputs the color, color intensity, and range for the task that has been issued, and is presented as the entered content and task. This mode compares the makeup contents and evaluates the comparison result. FIG. 16 is a flowchart illustrating an example of a processing procedure for the entire task mode. In the task mode process shown in FIG. 16, the point extraction in the face image extraction described above is performed (S41), the face standing analysis described above is then performed (S42), and the task mode process is performed (S43).

  Here, the specific processing content of the task mode shown in S43 will be described using a flowchart. FIG. 17 is a flowchart illustrating an example of specific processing contents of the task mode illustrated in S43. In the process shown in FIG. 17, it is first determined whether or not to select a situation (S51). If no situation is selected (NO in S51), the process is repeated until a situation is selected. If no input is made for a predetermined time, the task mode process may be terminated.

  When selecting a situation (YES in S51), it is next determined whether or not a makeup site is selected (S52). If a makeup site is not selected (NO in S52), the process of S52 is repeated until it is selected. If no input is made for a predetermined time, the task mode process may be terminated. In the process of S52, when a makeup part is selected (YES in S52), a makeup method is input (S53). Specifically, one of the pre-captured face images is selected, and the situation selected in S51 (for example, business, date, wedding, spring / summer, etc.) is selected and selected in S52. A color palette or the like is displayed on one or a plurality of parts (for example, base color, cheek, eyebrows, lipstick, etc.), and makeup is applied to the area designated by the user.

  Next, it is determined whether or not the input of the makeup method has been completed (S54). If the input of the makeup method has not been completed (NO in S54), the process returns to the process of S52 to perform the selection from the makeup site. In addition, when the input of the makeup method is completed in the process of S54 (YES in S54), a task determination is performed on the input content (S55). Specifically, using a preset determination table or the like, the color, color density, and range are evaluated at preset locations such as a base color, teak, and lipstick, and a determination is made based on the result. Although not shown, the color density can be input by, for example, displaying a gauge indicating the color density on the touch panel and operating the gauge by touch input. Further, when inputting the color intensity, a screen in which colors such as base color, teak, and lipstick displayed on the display screen in real time may be changed at any time may be displayed.

  Next, it is determined whether to end the assignment mode (S56). If the assignment mode is not ended (NO in S56), the process returns to S51 and the subsequent processes are repeated. If the assignment mode is to be ended (YES in S56), the assignment mode process is ended.

<About judgment table>
Here, a specific example of the above-described determination table will be described with reference to the drawings. FIG. 18 is a diagram illustrating an example of the determination table. In the determination table shown in FIG. 18, different types (types A, B, C, and D in the example of FIG. 18) are set for each situation, and “base color” and “eye shadow” in each type. , “Mascara”, “Teak”, “Lipstick”, “Lip Gloss”, “Eyebrow”, etc., the standard color (RGB), color density, range, etc. are stored in advance.

  At the time of determination, a plurality of determination values (for example, a table in which the criteria determined for each type as described above are digitized) as shown in FIG. 18 are provided for each situation. In the task mode, after selecting a situation in advance and applying makeup by free makeup, the game device 10 makes a decision, and the device side makes the decision table and the contents of makeup (color, position) actually entered by the user , Size (range)) and evaluating the distance from the value defined in the table, the correctness of the user's makeup method is determined.

  For example, when the predetermined value is 200 as the determination value and the value selected by the user is x, “x = 190 <x <210 → determined to be great!”, “X = 100 <x <190 or 210 <X <245 → determined as subtle ”and“ x = 0 <x <100 → determined as NG! ”. The above determination result can be expressed by characters as described above, but the result can also be calculated and displayed as a score, and the scoring method can be calculated for each part. In addition, the total points may be calculated and displayed. Moreover, according to the result, you may give the item arrange | positioned in the virtual space for avatars in the avatar room mentioned later.

<Free make mode>
Next, a screen transition diagram in the free make mode will be described with reference to the drawings. FIG. 19 is a diagram illustrating an example of a screen transition of the free make process. The face image displayed on the screen in FIG. 19 is actually a color face image acquired in advance by photographing or the like. In the free makeup shown in FIG. 19, one face data (face image) to be made is selected from the face images stored in the storage means. For example, “base”, “brow”, “eye shadow”, “eye” At least one part to be applied is selected from each part such as “line”, “mascara”, “lipstick”, “blusher”, “lip gloss”, and the like is applied to the selected part. At this time, by combining accessories, glasses, hairstyles, hats, and the like as play items, various makeups that cannot be actually displayed can be virtually displayed on the screen.

  In addition, although the area | region with respect to each part mentioned above is based on the position (area | region) extracted by each point extraction process in the face image extraction process mentioned above, for example, a lipstick is painted by shifting the point intentionally. By making the area larger or smaller than the actual size of the lips or changing the position and size of the eyebrows, post-makeup images of all patterns can be generated, and entertainment can be improved.

  In addition, on one screen of the game apparatus 10 (for example, the left screen), preset text content for displaying the character as if speaking is stored in a storage unit or the like. Here, FIG. 20 is a diagram illustrating an example of text (sentence) displayed during the game. As shown in FIG. 20, for example, “in the case of no makeup”, “text during the time when the user is applying makeup”, “text corresponding to the number of makeup parts”, “other various texts”, etc. A predetermined message can be displayed according to the condition (condition (1)) and the contents of the condition (2) set when the condition occurs. In addition, the content of the message displayed in this embodiment is not limited to the content shown in FIG.

<Other contents>
In addition to the above, the following functions can be provided as game content related to the makeup simulation described above.
(1) STEP UP BEAUTY
It may have functions to explain makeup, skin care, and basics of beauty care, techniques, and technical terms using illustrations and animations as well as text and facial scanning. In this case, for example, explanation of terms by dictionary function, registration of basics and techniques that should not be forgotten by bookmark function, make-up by beauty challenge function, skincare, preparation from basic to application of face care and quiz using touch pen , Review and so on.
(2) Avatar room As a rewarding healing mode such as training or beauty challenge passing, it is possible to create and display a 3D avatar having the same face as the face image using the face analysis described above. At this time, the ROM in the memory card stores the number of avatars corresponding to the type of face analysis, and the avatar extracted based on the result of the face analysis is stored in a room in a preset virtual space. Can act freely and heal users. Also, by providing such content that avatar items (clothes and furniture) can be obtained and events can occur as the game progresses, the entertainment in the makeup simulation is improved. be able to. In addition, the avatar may be set to repeat a specified movement, or may be moved by the user operating the avatar. Each avatar is created based on the user's own facial analysis and is an avatar having a face similar to that of the user, so that it is easy to get close, and therefore, it is a factor that prompts the user to play the game regularly.

  As described above, according to the present invention, it is possible to easily simulate the makeup desired by the user on the face image. In addition, according to the present invention, it is possible to easily analyze one's face and confirm a makeup face based on the result without using a specialized device in a cosmetic corner such as a department store, a beauty salon, a counter, etc. Can do. Further, it is possible to learn a method for making a makeup face most suitable for the person based on objective information.

  In addition, according to the present invention, it is easy for the person to analyze his / her face, and to carry out and determine the task given to the makeup method of the part of his / her face based on the result. The appropriate makeup face can be confirmed repeatedly.

  Since the present invention can be applied to, for example, a portable game device, for example, an object for performing a makeup simulation is not limited to a woman, but can be applied to an animal such as a man, a child, a dog, a cat, or the like. The software can be provided as a more entertaining software.

  In the present invention, the face image before makeup (whole or each part) and the face image after makeup are displayed using two screens, and the difference between them is displayed so that the user can easily understand the difference. Also good. Furthermore, colors suitable for TPO (Time (time), Place (location), and Occasion (case)) may be presented, and the skin color and the person's favorite color are registered, and color diagnosis is performed. Color determination can also be performed based on the above.

  In addition, it is possible to have a communication function in the game device. Through the communication function, a makeup method adapted to the season and the fashion of the year is stored at any time, and the makeup method adapted to the fashion of the year and the season is stored. Can also be provided.

  Moreover, in this application, although the touch panel used the method of 1 point detection, you may use another point detection panel. Further, the message is displayed visually by characters, but may be indicated by voice.

  While the invention has been described with reference to illustrative examples, various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as defined in the claims. Obviously it can be done. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings. The above embodiments are for understanding the present invention, and the present invention is not limited thereto.

1 is a perspective view of a game machine showing a first embodiment of the present invention. It is a figure which shows an example of the block configuration in a game machine. It is a figure which shows one structural example of the memory map of RAM in this embodiment. It is a flowchart which shows an example of the face image extraction process in this embodiment. It is a figure which shows an example of the edge extraction in this embodiment. It is a figure which shows an example of an HSV decomposition | disassembly image. It is a figure which shows an example of a mat key. It is a figure which shows an example of the game screen transition of the game software in this embodiment. It is a figure which shows an example of the whole process flowchart in recommended makeup. It is a flowchart which shows an example of a virtual makeup process. It is FIG. (1) which shows an example of the screen transition in recommended makeup processing. It is FIG. (2) which shows an example of the screen transition in recommended makeup processing. It is FIG. (1) which shows an example of the various information used in this embodiment. It is FIG. (2) which shows an example of the various information used in this embodiment. It is FIG. (3) which shows an example of the various information used in this embodiment. It is FIG. (1) which shows an example of the screen transition in situation makeup. It is FIG. (2) which shows an example of the screen transition in situation makeup. It is a figure which shows an example of the screen transition in a trend makeup. It is a flowchart which shows an example of a makeup commentary process. It is a flowchart which shows an example of the process sequence of the whole problem mode. It is a flowchart which shows an example of the specific processing content of the task mode shown to S43. It is a figure which shows an example of the determination table. It is a figure which shows an example of the screen transition of a free makeup process. It is a figure which shows an example of the text (sentence) displayed while a game is progressing.

Explanation of symbols

10 Game machines (game devices)
12 First display screen 14 Second display screen 16 Housing 18 Sound release hole 20 Operation switch 22 Touch panel 24 Stick or stylus pen (pen)
26 Storage 28 Imaging device (lens camera)
30 Memory Card 32 Insertion Port 40 Circuit Board 42 CPU Core 44 Bus 46 First Connector 48 Second Connector 50 RAM
52 First GPU
54 Second GPU
56 I / F circuit 58 Controller 60 First VRAM
62 Second VRAM
64 registers 70 program storage area 72 data storage area 80 game surface processing program 82 controller input detection program 84 touch panel input detection program 86 image generation program 88 main image display program 90 sub-screen display program 92 player character display control program 94 object display control program 100 Image data 102 Sound data 104 Object data buffer area 106 Operation data buffer area 108 Coordinate position data buffer area

Claims (8)

A game device capable of executing a game program that displays a virtual makeup image by combining a preset object in response to a user operation with the acquired face image,
Storage means for storing a plurality of makeup information in advance;
Imaging means for acquiring the face image;
Face image extraction means for extracting facial feature information consisting of the position and size of each part of the face from the face image obtained by the imaging means;
In accordance with the face feature information extracted by the face image extraction means, specific makeup face information is selected from a plurality of makeup face information accumulated in advance from the accumulation means, and the makeup face information is selected as the face image extraction means. A makeup face generating means for generating a post-makeup image adapted to the face image based on the facial feature information obtained by:
Display means for displaying the post-makeup image obtained by the makeup face generation means,
The face image extraction means includes
Edge extraction means for extracting an edge for at least one or more parts set in advance in the face image;
By combining the image obtained by the edge extraction means using a mask image generated from skin color determination based on the face image obtained by decomposing a color space into hue, saturation, and brightness, respectively, Mask extracting means for extracting;
The outer periphery of the region of the part extracted by the mask extraction unit is positioned by a plurality of preset points, and at least one of the positioned points is displayed on the display unit together with the enlarged image of the part, and the operation input and wherein said moving at least one point on the basis of the movement instructions for at least one point by means game apparatus characterized by comprising an adjusting means for adjusting the area where virtually performing cosmetic corresponding to the site . Classification means for positioning to any one of a plurality of face classifications preset from the face feature information extracted by the face image extraction means;
The makeup face generation means selects specific makeup face information from a plurality of pre-stored makeup face information according to the face feature information set for each face classification obtained by the classification means, and extracts the face image The game apparatus according to claim 1, wherein a post-makeup image adapted to the face image is generated based on the face feature information obtained by means.   Control is performed to correct the position information to be included in the makeup face information in accordance with the facial feature information including the position and size of each part obtained by the face image extraction means, and to store the corrected position information in the storage means. The game apparatus according to claim 1, further comprising a control unit configured to perform the control. The makeup face generation means includes
4. The game apparatus according to claim 1, further comprising advice presenting means for generating advice in accordance with the displayed makeup in addition to the displayed post-makeup image. A determination unit that provides a task suitable for the face image, and that determines the achievement level of the task from information input to the face image by the user in response to the task;
The game apparatus according to claim 1, further comprising: a determination display control unit that displays a determination result of the achievement level of the task determined by the determination unit on the display unit. A game device control method capable of executing a game program for displaying an image obtained by virtually combining a pre-set object in response to a user operation with an acquired face image. ,
An accumulation process for accumulating a plurality of makeup information in advance;
An imaging step of acquiring the face image;
A face image extraction step of extracting facial feature information consisting of the position and size of each part of the face from the face image obtained by the imaging step;
In accordance with the face feature information extracted in the face image extraction step, specific makeup face information is selected from a plurality of makeup face information accumulated in advance in the accumulation step, and the makeup face information is selected in the face image extraction step. A makeup face generation step of generating a post-makeup image adapted to the face image based on the facial feature information obtained by:
A display step of displaying on the display means for displaying the post-makeup image obtained by the makeup face generation step,
The face image extraction step includes
An edge extraction step of extracting an edge for at least one or more parts set in advance in the face image;
By combining the image obtained by the edge extraction step using a mask image generated from skin color determination based on the face image obtained by decomposing a color space into hue, saturation, and brightness, respectively, A mask extraction process to extract;
The outer periphery of the region of the part extracted by the mask extraction step is positioned by a plurality of preset points, and at least one of the positioned points is displayed on the display unit together with the enlarged image of the part, and the operation input and wherein said moving at least one point on the basis of the movement instructions for at least one point by means, the game control, characterized in that it comprises an adjusting step of adjusting the area where virtually performing makeup corresponding to the site Method. A game apparatus capable of executing a game program for synthesizing a preset object in response to a user operation and displaying a virtual makeup image on a display unit in response to a user operation is configured. Computer
Storage means for storing a plurality of makeup information in advance;
Imaging means for acquiring the face image;
Face image extraction means for extracting facial feature information consisting of the position and size of each part of the face from the face image obtained by the imaging means; and
In accordance with the face feature information extracted by the face image extraction means, specific makeup face information is selected from a plurality of makeup face information accumulated in advance from the accumulation means, and the makeup face information is selected as the face image extraction means. Based on the facial feature information obtained by the above, make it function as a makeup face generation means for generating a post-makeup image adapted to the face image,
The face image extraction means includes
Edge extraction means for extracting an edge for at least one or more parts set in advance in the face image;
By combining the image obtained by the edge extraction means using a mask image generated from skin color determination based on the face image obtained by decomposing a color space into hue, saturation, and brightness, respectively, Mask extracting means for extracting;
The outer periphery of the region of the part extracted by the mask extraction unit is positioned by a plurality of preset points, and at least one of the positioned points is displayed on the display unit together with the enlarged image of the part, and the operation input Game control, comprising: adjusting means for adjusting a region for virtually applying makeup corresponding to the part by moving the at least one point based on a movement instruction to the at least one point by the means program.   A computer-readable recording medium on which the game control program according to claim 7 is recorded.