JP7436872B2 - Server, recommendation system, and recommendation method - Google Patents

Description

The present technology relates to a server, a recommendation system , and a recommendation method, and particularly relates to a server, a recommendation system , and a recommendation method that enable more appropriate recommendations.

In recent years, many systems have been provided that use the user's facial image or answer questions to process the facial image with the user's desired makeup or output advice. There is.

Patent Document 1 discloses that a database in which color information and product information of cosmetics are registered is searched for the color information and product information of the cosmetic that is most suitable for the skin color of the face area in the user image taken with the user terminal, and the color information and product information of the cosmetic are searched. A makeup advice device is disclosed that sends makeup advice to a user terminal.

Japanese Patent Application Publication No. 2010-211308

As in the system of Patent Document 1, there is a possibility that appropriate recommendations cannot be made using facial images taken by users using their own mobile terminals.

The present technology was developed in view of this situation, and is intended to enable more appropriate recommendations to be made.

The server according to the first aspect of the present technology generates a photographed image in which a plurality of subjects are captured from a photo creation game machine that generates a photographic image by photographing a plurality of people as subjects in a photographing space having a camera and a lighting device. a receiving section for receiving;
Receive selection information representing the user's own face image from among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user , and receive the selection information representing the face image of the user himself/herself from among the face images extracted from the photographed image linked to the user; The server includes a generation unit that generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing the face image .
The recommendation system according to the first aspect of the present technology includes a photo creation game machine that generates a photographed image showing multiple subjects by photographing multiple people as subjects in a photography space equipped with a camera and a lighting device; Receive selection information representing the user's own face image among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user, and receive the face represented by the selection information. The recommendation system includes a server that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by analyzing images .

In the recommendation method of the first aspect of the present technology, a photo creation game machine takes pictures of multiple people as subjects in a photography space equipped with a camera and a lighting device, thereby generating a photographed image showing multiple subjects. receives, from a mobile terminal owned by the user, selection information representing the user's own face image among the face images extracted from the photographed image linked to the user, and displays the face image using the selection information. This recommendation method generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing the face image .

The second aspect of the present technology is a recommendation system that includes a photo creation game machine that generates a photographed image showing multiple subjects by photographing multiple people as subjects in a photographic space equipped with a camera and a light source, and a photo creation game machine owned by the user. receive selection information representing the user's own face image among the face images extracted from the photographed image linked to the user from a mobile terminal, and receive the face image represented by the selection information. a server that supplies recommendation information for recommending at least one of a product and a service to the user to the mobile terminal of the user based on an analysis result obtained by performing an analysis of It is a system.

In this technology, a photographed image is generated by photographing multiple people as subjects in a photographing space equipped with a camera and a lighting device, and from the photographed image linked to the user from a mobile terminal owned by the user. Selection information representing the user's own face image among the extracted face images is received, and based on an analysis result obtained by performing an analysis on the face image represented by the selection information , Recommendation information for recommending at least one of goods and services to the user is generated, or the recommendation information is supplied to the mobile terminal of the user.

A server according to a first aspect of the present technology includes a receiving unit that receives a photographed image from a photo creation game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a lighting device; A server comprising a generation unit that generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing a subject area of a photographed image. be.
A recommendation system according to a first aspect of the present technology includes a photo creation game machine that generates a photographed image by photographing a user as a subject in a photographic space having a camera and a lighting device, and an analysis of the subject area of the photographed image. The recommendation system includes a server that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by the analysis.

Note that the effects described here are not necessarily limited, and may be any of the effects described in this disclosure.

1 is a diagram illustrating a configuration example of a recommendation system. FIG. 2 is a block diagram showing an example of the hardware configuration of a photo creation game machine. FIG. 2 is a block diagram showing an example of the hardware configuration of a server. FIG. 2 is a block diagram showing an example of the hardware configuration of a mobile terminal. FIG. 2 is a diagram showing an example of the internal configuration of a shooting space of a photo creation game machine. FIG. 3 is a diagram illustrating a flow of presenting recommendation information based on a face image. FIG. 2 is a block diagram showing an example of a functional configuration of a recommendation system. FIG. 3 is a diagram showing an example of user data. It is a diagram explaining the flow of operation of the recommendation system. FIG. 7 is a diagram illustrating an example of a live view screen when photographing for face analysis. FIG. 7 is a diagram showing an example of a live view screen during normal shooting. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram showing an example of a recommended site screen displayed on a mobile terminal. FIG. 3 is a diagram illustrating a flow of presenting recommendation information based on a whole-body image.

Below, a mode for implementing the present technology (hereinafter referred to as an embodiment) will be described. The explanation will be given in the following order.

1. Configuration of recommendation system 2. Shooting in the shooting space of a photo creation game machine 3. Presentation of recommendation information based on face images 4. Variations

<1. Recommendation system configuration>
(Summary of recommendation system)
FIG. 1 is a diagram showing an overview of a recommendation system according to an embodiment of the present technology.

The recommendation system 1 shown in FIG. 1 includes a photo creation game machine 10, an image management server 20, a recommendation server 30, and mobile terminals 40-1 and 40-2.

Each of the devices constituting the recommendation system 1 is directly or indirectly connected to the network NW and communicates with each other. The network NW includes the Internet and a local area network, and may be configured with a wired communication line or a wireless communication line.

Further, in the recommendation system 1, the image management server 20 and the recommendation server 30 are each configured separately, but they may be configured as one.

The photo creation game machine 10 is installed in a store such as a game center or an amusement facility, and provides a photo creation game to the user by accepting coins for the play fee. Specifically, the photo creation game machine 10 accepts an editing operation (doodle) on a photographed image obtained by photographing a user, and provides the photographed image to the user by printing it on sticker paper.

Users of the photo creation game machine 10 are mainly high school girls and young women. In the photo creation game machine 10, mainly two users per group can enjoy the game. Of course, in the photo creation game machine 10, three or more users or a single user can enjoy the game.

Although only one photo creation game machine 10 is shown in FIG. 1, a plurality of photo creation game machines 10 actually exist in the recommendation system 1.

The image management server 20 manages captured images obtained by shooting with the photo creation game machine 10, and becomes a member of a website (image acquisition site) operated by a manufacturer that manufactures and sells the photo creation game machine 10. Manage user information.

A user who has become a member can view captured images obtained by shooting with the photo creation game machine 10 on the mobile terminal 40-1 or the mobile terminal 40-2.

The image management server 20 acquires, for each user who has become a member, a captured image obtained by shooting with the photo creation game machine 10 from the photo creation game machine 10 (such as a store where the photo creation game machine 10 is installed). do. Photographed images for each user can be supplied to the recommendation server 30 as necessary.

The recommendation server 30 provides a website (recommendation site) that recommends products and services to each user using captured images for each user from the image management server 20. Specifically, the recommendation server 30 generates recommendation information for recommending products and services based on analysis results obtained by analyzing captured images of each user, and generates recommendation information for recommending products and services. 40-1 and the mobile terminal 40-2.

The mobile terminals 40-1 and 40-2 are configured by a mobile phone (smartphone), a mobile PC (personal computer), a tablet terminal, etc. owned by each user. In the following, when the mobile terminals 40-1 and 40-2 are not distinguished from each other, they are simply referred to as the mobile terminal 40. Note that although there are only two mobile terminals 40 illustrated in FIG. 1, a plurality of mobile terminals 40 actually exist in the recommendation system 1.

(Hardware configuration of photo creation game machine)
FIG. 2 is a block diagram showing an example of the hardware configuration of the photo creation game machine 10. As shown in FIG.

As shown in FIG. 2, in the photo creation game machine 10, a pre-selection section 102, a photographing section 103, an editing section 104, and a printing section 105 are connected to a PC section 101 via an external input/output interface 106. It is composed of

A CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, and a RAM (Random Access Memory) 113 that constitute the PC section 101 are interconnected by a bus 114.

The CPU 111 executes a predetermined program and controls the overall operation of the photo creation game machine 10. The ROM 112 stores programs and data executed by the CPU 111. The RAM 113 temporarily stores data and programs processed by the CPU 111.

An input/output interface 115 is further connected to the bus 114. A storage section 116, a communication section 117, and a drive 118 are connected to the input/output interface 115.

The storage unit 116 is a nonvolatile storage medium such as a hard disk or flash memory. The storage unit 116 stores various setting information and the like supplied from the CPU 111. Information stored in the storage unit 116 is read out by the CPU 111 as appropriate.

The communication unit 117 is an interface for a network such as the Internet. The communication unit 117 communicates with an external device under the control of the CPU 111. The communication unit 117 transmits the image selected by the user to the image management server 20. The image transmitted from the communication unit 117 is allocated a predetermined storage area and saved, and is displayed on the mobile terminal 40 that has accessed the image acquisition site or downloaded to the mobile terminal 40.

A removable medium 119 such as an optical disk or a semiconductor memory is appropriately installed in the drive 118. Programs and data read from the removable medium 119 by the drive 118 are supplied to the CPU 111 and stored or installed in the storage unit 116.

An external input/output interface 106 is connected to the input/output interface 115 . Control of each section by the PC section 101 is performed via an external input/output interface 106.

The pre-selection unit 102 performs pre-selection processing for users before photographing. The advance selection section 102 includes a touch panel monitor 121, a speaker 122, and a coin processing section 123.

The touch panel monitor 121 displays various selection screens under the control of the PC unit 101 and receives user operations on the selection screens. An input signal representing the content of the user's operation is supplied to the PC section 101, and various settings are performed.

The speaker 122 outputs voice guidance for operations on various selection screens, BGM (background music), sound effects, and the like.

The coin processing unit 123 detects insertion of a coin into the coin insertion slot. When the coin processing unit 123 detects that a predetermined amount of coins have been inserted, it outputs an activation signal to the PC unit 101 instructing the start of the game.

The photographing unit 103 performs photographing processing for a user who is in a photographing space formed within the housing of the photo creation game machine 10 . The photographing unit 103 includes a camera 131, a touch panel monitor 132, a lighting device 133, and a speaker 134.

The camera 131 performs photography according to shutter control by the PC unit 101 and outputs image data obtained by photography to the PC unit 101.

Touch panel monitor 132 is provided near camera 131. The touch panel monitor 132 has a function as a live view monitor that displays moving images captured by the camera 131, and a function that displays various GUIs and receives selection operations from the user.

The illumination device 133 is a strobe unit provided in the photographing space, and emits light in accordance with the illumination control signal supplied from the PC section 101.

The speaker 134 is provided at a predetermined position within the shooting space, and outputs guidance audio, BGM, sound effects, etc. for the shooting process.

The editing unit 104 performs editing processing after shooting. The editing section 104 includes a tablet built-in monitor 141, touch pens 142A, 142B, and a speaker 143.

The tablet built-in monitor 141 and the touch pens 142A and 142B are provided on each of the two surfaces of the housing that constitutes the editing section 104. When there is no need to distinguish between the touch pen 142A and the touch pen 142B, they are simply referred to as the touch pen 142.

The tablet built-in monitor 141 displays an editing screen under the control of the PC unit 101 and receives user operations using the touch pen 142 on the editing screen. An input signal representing the content of the user's operation is supplied to the PC section 101, and the photographed image to be edited is edited.

The speaker 143 is built into a part of the casing that constitutes the editing unit 104, and outputs guide audio for editing processing, BGM, sound effects, and the like.

The printing unit 105 is configured to include a printer 151. A sticker paper unit 152 is attached to the printer 151.

The printer 151 prints an image on sticker paper stored in the sticker paper unit 152 based on print data supplied from the PC section 101. The printer 151 discharges the sticker paper on which the image is printed to a sticker paper discharge port.

(Hardware configuration of image management server/recommendation server)
FIG. 3 is a block diagram showing an example of the hardware configuration of the server 200 that constitutes the image management server 20 and the recommendation server 30.

In FIG. 3, a CPU 211 executes various processes according to a program stored in a ROM 212 or a program loaded into a RAM 213. The RAM 213 also appropriately stores data necessary for the CPU 211 to execute various processes.

CPU 211 , ROM 212 , and RAM 213 are interconnected via bus 214 . An input/output interface 215 is also connected to this bus 214.

An input section 216 , an output section 217 , a storage section 218 , and a communication section 219 are connected to the input/output interface 215 .

The input unit 216 includes a keyboard, a mouse, and the like. The output unit 217 includes a display made of liquid crystal or organic EL (Electro-Luminescence), a speaker, and the like. The storage unit 218 is composed of a hard disk or the like. The communication unit 219 includes a wired communication module that performs wired communication, a wireless communication module that performs wireless communication, and the like.

The communication unit 219 performs communication processing via the network NW.

A drive 220 is also connected to the input/output interface 215 as necessary, and a removable medium 221 such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc. is appropriately installed. Computer programs read from them are installed in the storage unit 218 as necessary.

(Hardware configuration of mobile terminal)
FIG. 4 describes an example of the hardware configuration of the mobile terminal 40.

In FIG. 4, a CPU 411 executes various processes according to a program stored in a ROM 412 or a program loaded into a RAM 413. The RAM 413 also appropriately stores data necessary for the CPU 411 to execute various processes.

CPU 411, ROM 412, and RAM 413 are interconnected via bus 414. An input/output interface 415 is also connected to this bus 414.

An input section 416 , an output section 417 , a storage section 418 , and a communication section 419 are connected to the input/output interface 415 .

The input unit 416 includes keys, buttons, a touch panel, a camera, a microphone, and the like. The output unit 417 includes a display made of liquid crystal, organic EL, etc., formed together with a touch panel, and a speaker. The storage unit 418 is configured of, for example, a NAND flash memory. The communication unit 419 includes a communication module that performs wireless communication.

<2. Shooting in the shooting space of a photo creation game machine>
Here, shooting in the shooting space of the photo creation game machine 10 will be explained.

FIG. 5 is a diagram showing an example of the internal configuration of a shooting space formed within the housing of the photo creation game machine 10.

In the photographing space, the user faces the camera unit CU, which includes a camera 131 and a touch panel monitor 132, to take a photograph. In addition to the above-described camera unit CU, on the front side of the user in the shooting space (on the left side in FIG. 5), as a lighting device 133, there is an upper strobe unit 133-1 above the camera unit CU, and a foot strobe unit below the camera unit CU. 133-2 is provided.

Further, in the photographing space, left and right strobe units 133-3 are provided on the left and right upper sides when viewed from the user who directly faces the camera unit CU.

Furthermore, although not shown, in the photographing space, a rear strobe unit may be provided above the back of the user directly facing the camera unit CU, or a ceiling strobe unit may be provided on the ceiling of the photographing space.

Each strobe unit (illumination device 133) provided in the photographing space in this manner has a strobe light emitting tube therein, and emits strobe light at the photographing timing of the camera 131.

Specifically, the upper strobe unit 133-1 emits strobe light L1 from diagonally above the front of the user, focusing on the user's face, at the timing when the camera 131 takes a picture. The foot strobe unit 133-2 emits strobe light L2 from diagonally below the front of the user, focusing on the lower body from the feet at the timing of the camera 131 taking a picture. The left and right strobe units 133-3 emit strobe light L3 from diagonally above the left and right sides of the user's head, focusing on the left and right temporal regions, at the timing when the camera 131 takes a picture.

Additionally, inside the upper strobe unit 133-1, the foot strobe unit 133-2, and the ceiling strobe unit (not shown), in addition to strobe light tubes, there are constant light sources such as LED (Light Emitting Diode) bulbs. provided. Thereby, the brightness in the photographing space can be maintained constant in addition to the photographing timing.

That is, in the photographing space of the photo creation game machine 10, photographing is performed in an optical environment such as a constant amount of light and a constant light emission time.

Furthermore, before the timing of photographing by the camera 131, a live view screen including a live view image captured by the camera 131 is displayed on the touch panel monitor 132. On the live view screen, instruction information that instructs the user about the standing position and the direction of the face is presented to the user.

That is, in the photographing space of the photo creation game machine 10, photographs are taken while keeping the size and orientation of the face constant relative to the angle of view.

Furthermore, the white balance, brightness, hue, angle of view, etc. of the camera 131 are adjusted in advance for each photo creation game machine 10 in order to suppress variations in hue, brightness, angle of view, etc. These adjustments are made so that the values of each parameter fall within a certain range.

That is, in the photography space of the photo creation game machine 10, photography is performed in a state where individual differences between the cameras 131 of each photo creation game machine 10 are suppressed as much as possible.

As described above, in the shooting space of the photo creation game machine 10, shooting is performed with constant shooting conditions such as the optical environment, the size and orientation of the face relative to the angle of view, and individual differences in the camera 131, so the amount of light is It is possible to stabilize the appearance of the photographed image (especially the appearance of the user's face area) obtained by the photographing without any fluctuation.

Note that photography by the camera 131, display on the touch panel monitor 132, and light emission from each strobe unit (lighting device 133) are all controlled by the CPU 111 (FIG. 2).

<3. Presentation of recommendation information based on facial images>
In the following, an example will be described in which recommendation information is presented to the user based on a face image extracted from a photographed image with a stable appearance.

(Flow of presentation of recommendation information)
FIG. 6 is a diagram illustrating the flow of presenting recommendation information based on facial images.

In FIG. 6, a photo creation game machine 10, a recommendation server 30, and a mobile terminal 40 that constitute the recommendation system 1 are shown.

In the photo creation game machine 10, as described above, a stable photographed image can be obtained by photographing the user who owns the mobile terminal 40 under constant photographing conditions. From the photo creation game machine 10, the face image P11 extracted from the photographed image is supplied to the recommendation server 30 via the image management server 20. The photographed image may be directly supplied from the photo creation game machine 10 to the recommendation server 30, and the facial image P11 may be extracted from the photographed image at the recommendation server 30.

The recommendation server 30 analyzes the face image P11. Then, the recommendation server 30 generates recommendation information for recommending products and services to the user based on the obtained analysis results. For example, recommendation information R31 for recommending makeup and cosmetics that match the user's face is generated based on the analysis result of the face image. The recommendation server 30 transmits the generated recommendation information R31 to the mobile terminal 40 owned by the user.

However, the present invention is not limited to this, and the photo creation game machine 10 may analyze the face image P11 and supply the obtained analysis result to the recommendation server 30 via the image management server 20 together with the face image P11. In this case, the recommendation server 30 generates recommendation information R31 based on the analysis result from the photo creation game machine 10.

Further, the photo creation game machine 10 analyzes the face image P11, generates recommendation information R31 based on the obtained analysis result, and sends the generated recommendation information R31 to the image management server 20 along with the face image P11. The information may also be supplied to the recommendation server 30 via the service provider. In this case, the recommendation server 30 transmits the recommendation information R31 from the photo creation game machine 10 to the mobile terminal 40.

In the above-described flow, recommendation information based on the user's face image can be presented to the user on the mobile terminal 40.

(Example of functional configuration of recommendation system)
FIG. 7 is a block diagram showing an example of the functional configuration of the recommendation system 1 that realizes the presentation of recommendation information based on the above-mentioned facial images.

In FIG. 7, the photo creation game machine 10, image management server 20, recommendation server 30, and mobile terminal 40 that constitute the recommendation system 1 are shown. The explanation of is omitted.

The photo creation game machine 10 includes a control section 510 in addition to the camera 131, touch panel monitor 132, lighting device 133, and printer 151 described above. Each functional block constituting the control unit 510 is realized by the CPU 111 executing a predetermined program.

The control section 510 is configured to include a photographing processing section 511, an image processing section 512, and a printing processing section 513.

The photographing processing unit 511 executes photographing processing for the user by controlling a camera 131, a touch panel monitor 132, and a lighting device 133 provided in the photographing space.

The image processing unit 512 performs various types of image processing on a plurality of captured images.

For example, the image processing unit 512 performs predetermined image processing on the subject area of the captured image. Specifically, the image processing unit 512 processes (enlarges/reduces, deforms, changes color, etc.) and retouches the outline of the user's face and facial parts such as eyes, nose, mouth, and cheeks that appear in the captured image. Apply retouching to the image. Further, the image processing unit 512 performs editing processing on the photographed image according to the user's editing operation (doodle).

Editing processing is performed on the captured image selected by the user according to the user's editing operation, while retouching processing is basically applied uniformly to all captured images regardless of the user's operation. It is applied to By performing such image processing, a processed image that has been subjected to image processing is generated separately from a captured image that has not been subjected to image processing.

The print processing unit 513 controls the printer 151 to print and output the processed image on sticker paper.

Further, the processed image is transmitted from the photo creation game machine 10 to the image management server 20, together with the photographed image that has not been subjected to image processing, and is managed for each user.

The recommendation server 30 includes a control unit 520 and a user DB 530. Each functional block constituting the control unit 520 and the user DB 530 are realized by the CPU 211 executing a predetermined program.

The control unit 520 is configured to include a web page providing unit 521, an image analysis unit 522, a recommendation information generation unit 523, and a data management unit 524.

The web page providing unit 521 creates a web page that constitutes a website (recommended site) to be viewed on the mobile terminal 40 and provides the web page to the mobile terminal 40 .

The image analysis unit 522 analyzes the subject area of the captured image in response to the acquisition of the captured image of the user corresponding to the mobile terminal 40 that accessed the recommended site from the image management server 20 . The analysis by the image analysis unit 522 is performed on captured images (original images) that have not been subjected to image processing among the captured images managed by the image management server 20 . Specifically, the image analysis unit 522 analyzes a face image extracted from a photographed image to be analyzed. Hereinafter, a captured image to be analyzed (a captured image that has not been subjected to image processing) will be referred to as an analysis image.

The recommendation information generation unit 523 generates recommendation information for the user corresponding to the face image based on the analysis result obtained by the image analysis unit 522 analyzing the analysis image (face image).

The data management unit 524 manages user data regarding users registered in the recommended site among the users corresponding to the mobile terminal 40 who have accessed the recommended site. User data is stored in user DB 530.

FIG. 8 is a diagram showing an example of user data.

As shown in FIG. 8, the user data d530 stored in the user DB 530 is configured by associating a user ID with a face image ID and one or more pieces of past recommendation information.

The user ID is user-specific identification information registered in the recommended site. The face image ID is identification information unique to the face image analyzed for the user specified by the user ID. Past recommendation information is recommendation information generated for a user specified by a user ID. Past recommendation information is added and accumulated, for example, each time a facial image specified by a facial image ID is analyzed.

The user data d530 having such a structure allows users registered on the recommendation site to view recommendation information presented in the past.

(Operation of recommendation system)
Referring to FIG. 9, the flow of operation of the recommendation system will be explained. The process in FIG. 9 starts when the photo creation game machine 10 starts providing the game to the user.

First, in step S11, the photo creation game machine 10 (photographing processing unit 511) executes a photographing process. In the photographing process, a plurality of photographed images are generated by performing photographing multiple times with the user as the subject.

Specifically, in the photographing process, for example, the first photograph among multiple photographs is performed as a face analysis photograph to generate an analysis image, and the second and subsequent photographs are performed as a face analysis photograph to generate an analysis image. This is performed as normal photography to generate a photographed image to be processed.

FIG. 10 is a diagram showing an example of a live view screen displayed on the touch panel monitor 132 when photographing for face analysis.

In the live view screen shown in FIG. 10, substantially square live view display areas 611A and 611B are provided side by side. At the bottom of the live view display areas 611A, 611B, there is a message indicating that face analysis photography is to be performed, as instruction information indicating the standing position and face direction, and inside the frame (live view display areas 611A, 611B). A message will be displayed prompting you to frame your face and face forward when taking the photo.

The two main users of the photo creation game machine 10 adjust their standing positions and face orientations so that their respective faces fit within the live view display areas 611A and 611B.

When the photographing timing comes in such a state, face analysis photographing is performed and an analysis image is generated.

On the other hand, FIG. 11 is a diagram showing an example of a live view screen displayed on the touch panel monitor 132 during normal shooting.

In the live view screen shown in FIG. 11, a vertically long rectangular live view display area 621 is provided at the center of the screen. On the left and right sides of the live view display area 621, sample pose images of a model in a sample pose as the subject are displayed every time multiple (for example, five) normal shootings are performed.

The two main users of the photo creation game machine 10 adjust their own poses and facial expressions while checking the sample pose images and the live view display area 621 displayed on the left and right sides of the live view display area 621. conduct.

When the shooting timing comes in such a state, normal shooting is performed and a shot image is generated.

A shooting result display area 622 is provided below the live view display area 621 in which a shot image generated by normal shooting is displayed. In the example of FIG. 11, five photographed images are displayed as the photographing results of five normal photographing operations.

As described above, when, for example, one image for analysis and five captured images are generated in the imaging process, the process advances to step S12, and the image processing unit 512 performs image processing on the captured images.

As described above, for example, five captured images are subjected to retouching processing without depending on the user's operation, and are subjected to editing processing in response to the user's editing operation. As a result, five processed images are generated. Here, it is assumed that the analysis image is also subjected to retouching processing. Hereinafter, the analysis image that has been subjected to retouching processing will be referred to as the processed analysis image.

In step S13, the print processing unit 513 controls the printer 151 to print and output at least one of the five processed images on a sticker sheet.

Thereafter, in step S14, the communication unit 117 (FIG. 2) of the photo creation game machine 10 transmits the five processed images and the analysis image (including the processed analysis image) to the image management server 20.

Note that a photographed image (original image) that is generated by normal photographing and that has not been subjected to image processing may be further transmitted to the image management server 20. In this case, instead of using the original image generated by face analysis photography as the analysis image, we analyze the original image generated by normal photography that can perform the face recognition necessary for facial image analysis. The image may be sent to the image management server 20 as an image for use.

The processed image sent to the image management server 20 is displayed on or downloaded to the mobile terminal 40 of the user who has become a member of the image acquisition site.

Now, in step S21, the communication unit 219 (FIG. 3) of the image management server 20 transmits the analysis image (including the processed analysis image) to the recommendation server 30.

Here, it is assumed that members registered in the image acquisition site provided by the image management server 20 and members registered in the recommendation site provided by the recommendation server 30 are linked by a predetermined method. For example, by using the email address of the mobile terminal 40 or the SNS (Social Networking Service) account as a common user ID, the user can register himself as the same member on both the image acquisition site and the recommendation site. .

Note that the timing at which the image for analysis is sent from the image management server 20 to the recommendation server 30 may be not every time the user plays a game, but may be every relatively long period, such as once a month, for example. . This is because the user's face that appears in the analysis image does not change significantly over time compared to the user's face that appears in the post-processing image that undergoes various types of image processing.

In this state, for example, when the user performs an operation on the login screen displayed by accessing the recommended site from the mobile terminal 40, in step S41, the communication unit 419 of the mobile terminal 40 connects the recommended site to the recommended site. A login request for logging in is sent to the recommendation server 30.

In step S31, the web page providing unit 521 of the recommendation server 30 accepts the login request from the mobile terminal 40, and displays a selection screen on the mobile terminal 40 for allowing the user to select the user himself from the subjects appearing in the analysis image. to be displayed.

Since an analysis image often includes a plurality of subjects, such as two people, a single analysis image can be associated with the plurality of users (members). In this case, even if it is possible to identify the analysis image associated with a certain user, it is not possible to identify which subject in the analysis image is the user.

Therefore, by having the user select himself or herself from the subjects in the analysis image, it is possible to specify which subject in the analysis image is the user. Specifically, a plurality of facial images extracted from the analysis image are displayed on the selection screen, and the user is prompted to select one of the facial images. In the following, it is assumed that two users are shown in the analysis image, and two facial images are displayed on the selection screen.

In step S42, the communication unit 419 of the mobile terminal 40 transmits selection information representing the selection (selected face image) to the recommendation server 30 in response to the user's operation on the selection screen.

In step S32, the image analysis unit 522 of the recommendation server 30 identifies which subject is the user of the mobile terminal 40 in the analysis image based on the selection information from the mobile terminal 40, and performs analysis on the subject area. conduct. Specifically, analysis is performed on the face image represented by the selection information among the plurality of face images extracted from the analysis image.

Next, in step S33, the web page providing unit 521 of the recommendation server 30 causes the mobile terminal 40 to display a question screen for allowing the user of the mobile terminal 40 to select a question and an answer to the question.

In step S43, the communication unit 419 of the mobile terminal 40 transmits, to the recommendation server 30, answer information representing the selection of the answer to the user's question in response to the user's operation on the question screen. Questions asked to the user include, for example, questions regarding the user's skin color and questions regarding makeup that the user usually wears.

Display of such a question screen and transmission of answer information are often performed multiple times.

After that, in step S34, the recommendation information generation unit 523 of the recommendation server 30 uses the analysis result obtained by the image analysis unit 522 analyzing the analysis image and the plurality of response information from the mobile terminal 40. Based on the information, recommendation information for the user of the mobile terminal 40 is generated. Here, recommendation information for recommending at least one of makeup-related products and services is generated, for example, based on the analysis result of the facial image and the answer information to the above-mentioned question.

In step S35, the web page providing unit 521 of the recommendation server 30 transmits a web page including recommendation information to the mobile terminal 40.

Then, in step S44, the output unit 417 (FIG. 4) of the mobile terminal 40 presents the web page containing the recommendation information from the recommendation server 30 to the user.

(Screen transition of recommended site)
Next, in the flow of the operation of the recommendation system 1 described above, screen transitions of the recommended site viewed by the user using the mobile terminal 40 will be explained. Here, an example of screen transitions from when a user registered as a member of a recommendation site logs into the recommendation site and receives recommendation information is shown.

First, when the mobile terminal 40 accesses the recommended site, a login screen 710 shown in FIG. 12 is displayed on the display (output unit 417) of the mobile terminal 40. The login screen 710 is provided with a user icon display area 711, a password input box 712, and a login button 713.

The user icon display area 711 is an area where icons set for each user of the mobile terminal 40 are displayed. The icon displayed in the user icon display area 711 may be a humanoid figure automatically set by the system as shown in FIG. 12, or may be a user's face image set by the user. .

Password input box 712 is a text box for the user to enter a password for logging into the recommended site. When the user presses the login button 713 with the correct password entered in the password input box 712, a login request is sent from the mobile terminal 40 to the recommendation server 30 (step S41 in FIG. 9), and the login is completed. .

When the login is completed, a selection screen 720 shown in FIG. 13 is displayed on the display of the mobile terminal 40. The selection screen 720 is provided with face image display areas 721A, 721B, selection buttons 722A, 722B, and a diagnosis start button 723.

The facial image display areas 721A and 721B are areas where facial images of two users extracted from the analysis images associated with the users of the mobile terminal 40 are displayed. The facial images displayed in the facial image display areas 721A and 721B are facial images extracted from the post-processing analysis image. That is, the face image display areas 721A and 721B display face images that have been subjected to retouching processing that processes or corrects the outline of the face, eyes, nose, mouth, cheeks, and the like.

Note that these facial images may be extracted from the post-processed analysis images in the recommendation server 30, or may be transmitted to the recommendation server 30 after being extracted from the post-processed analysis images in the photo creation game machine 10. good.

The selection buttons 722A and 722B are buttons for the user to select his or her own face from the face images displayed in the face image display areas 721A and 721B. When the selection button 722A is pressed, the face image displayed in the face image display area 721A is selected, and when the selection button 722B is pressed, the face image displayed in the face image display area 721B is selected.

When the user presses the diagnosis start button 723 with either the selection button 722A or 722B selected, selection information representing the selected facial image is transmitted from the mobile terminal 40 to the recommendation server 30 (Fig. 9 step S42).

In the recommendation server 30, analysis of the face image represented by the selection information is started (step S32 in FIG. 9). Specifically, feature points of a face in a face image are detected, and the face is mapped onto a preset feature amount space. A face type is set in each region of the feature space, and by mapping the face onto the feature space, the face type of the user is identified.

Here, the facial image to be analyzed is not a processed analysis image, but a facial image extracted from an analysis image (original image) that has not been subjected to image processing. Since the face image extracted from the post-processing analysis image has undergone deformation through retouching, etc., feature points cannot be detected correctly, and the originally required analysis cannot be performed. On the other hand, the face image extracted from the analysis image is not modified by retouching, and the feature points are detected correctly, so it is possible to perform the originally required analysis.

Now, when the diagnosis start button 723 is pressed, a diagnosis screen 730 shown in FIG. 14 is displayed on the display of the mobile terminal 40 as one of the above-mentioned question screens. The diagnosis screen 730 is a question screen for asking the user about the tendency of skin color and having the user answer the question. The diagnosis screen 730 is provided with answer buttons 731A, 731B and a next button 732.

The answer button 731A is a button for selecting that the skin color is more bluish as an answer to the question. The answer button 731B is a button for selecting that the skin color is more yellowish as an answer to the question.

When the user presses the next button 732 with either answer button 731A or 731B selected, answer information representing the selected answer is transmitted from the mobile terminal 40 to the recommendation server 30 (Fig. 9 step S43).

When the next button 732 is pressed, a diagnosis screen 740 shown in FIG. 15 is displayed on the display of the mobile terminal 40 as another one of the above-mentioned question screens. The diagnosis screen 740 is a question screen for asking the user whether or not to wear blush and asking the user to answer. The diagnosis screen 740 is provided with answer buttons 741A, 741B and a next button 742.

The answer button 741A is a button for selecting to apply blush as an answer to a question. The answer button 741B is a button for selecting not to apply blush as an answer to the question.

When the user presses the next button 742 with either answer button 741A or 741B selected, answer information representing the selected answer is transmitted from the mobile terminal 40 to the recommendation server 30 (Fig. 9 step S43).

In the recommendation server 30, recommendation information for the user of the mobile terminal 40 is generated based on the facial image analysis result and a plurality of pieces of response information from the mobile terminal 40 (step S34 in FIG. 9).

Thereafter, a diagnosis result screen 750 shown in FIG. 16 is displayed on the display of the mobile terminal 40. The diagnosis result screen 750 is a screen for presenting preliminary recommendation information generated by the recommendation server 30 at a stage before generating recommendation information. The preliminary recommendation information here is information representing features and characteristics for deriving the product or service represented by the recommendation information. The diagnosis result screen 750 is provided with a recommended preliminary information display area 751 and an item button 752.

The recommended preliminary information display area 751 is an area where the above-mentioned recommended preliminary information is displayed. In the example of FIG. 16, words and images representing a lip color (honey terracotta in the figure) that should be recommended to the user are displayed as preliminary recommendation information.

The item button 752 is a button for presenting recommendation information (products and services) derived based on the preliminary recommendation information displayed in the preliminary recommendation information display area 751.

That is, when the item button 752 is pressed, a diagnosis result screen 760 shown in FIG. 17 is displayed on the display of the mobile terminal 40. The diagnosis result screen 760 is a screen for presenting recommendation information generated by the recommendation server 30. The diagnosis result screen 760 is provided with a recommendation information display area 761 and evaluation buttons 762A, 762B, and 762C.

The recommended information display area 761 is an area where recommended information is displayed. In the example of FIG. 17, words and images representing the lipstick (product) corresponding to the color presented as the preliminary recommendation information are displayed as the recommendation information.

By selecting the recommended information display area 761 by tapping or the like, the user can access a purchase screen for the product displayed in the recommended information display area 761 and purchase the product.

Evaluation buttons 762A, 762B, and 762C are buttons for users to evaluate products and services presented as recommendation information (select preferences for products and services). Specifically, the evaluation button 762A is a button for highly evaluating a product presented as recommendation information. The evaluation button 762B is a button for evaluating the product presented as recommendation information as moderate. The evaluation button 762C is a button for giving a low evaluation to a product presented as recommendation information.

The recommendation information presented on the diagnosis result screen in this way is linked to the user ID as past recommendation information, along with evaluation information representing the user's evaluation of the product or service presented as the recommendation information, and preliminary recommendation information. , is stored in the user DB 530 as user data d530.

In addition, a user who logs in to the recommendation site from the mobile terminal 40 can check the recommendation information presented on the diagnosis result screen in the past (past recommendation information) by accessing the user's own management page, My Page. can.

On my page, in addition to viewing past recommended information, users can edit profile information such as the user's name, icon, and password on recommended sites, and check the history of purchased products. My page 770 shown in FIG. 18 is an example of my page when viewing past recommendation information.

My page 770 in FIG. 18 displays a diagnosis result list display area 771 and a past recommendation information display area 772.

The diagnosis result list display area 771 is an area where preliminary recommendation information presented as the most recent diagnosis result and answer information representing an answer to a question selected by the user are displayed. In the example of FIG. 18, text and an image representing the lip color recommended to the user are displayed as the preliminary recommendation information, and text and an image representing the skin color of the user's face are displayed as the answer information.

The past recommended information display area 772 is an area where recommended information (products and services) presented in the past is displayed. In the example of FIG. 18, a plurality of lipsticks (products) that have been recommended to the user in the past are displayed as recommendation information.

As described above, a photographed image is generated by photographing with constant photographing conditions in the photo creation game machine 10, and recommendation information is generated based on the analysis result of a face image extracted from the photographed image. Ru. In other words, the analysis is based on the analysis results of a stable facial image, rather than the situation where the amount of light fluctuates depending on the optical environment and other shooting conditions, resulting in unstable facial images, as occurs when taking pictures using a user's mobile device. Since recommendation information is generated based on the above information, it becomes possible to make more appropriate recommendations to the user.

In the above example, the recommendation information is presented when the user logs in to the recommendation site, but the present invention is not limited to this. For example, after the user logs in to the recommendation site, the product or By searching for a service, corresponding recommendation information may be presented.

<4. Modified example>
Below, a modification of the recommendation system to which the present technology is applied will be described.

(Presentation of recommendation information based on whole body image)
In the above, an example has been described in which recommendation information is presented to the user based on a face image extracted from a photographed image with a stable appearance.

However, the recommendation system to which the present technology is applied can also present recommendation information to the user based on a whole-body image of a person (whole-body image) extracted from a stable captured image.

FIG. 19 is a diagram illustrating the flow of presenting recommendation information based on a whole-body image.

In FIG. 19, a photo creation game machine 10, a recommendation server 30, and a mobile terminal 40 that constitute the recommendation system 1 are shown.

In the photo creation game machine 10, as described above, a stable photographed image can be obtained by photographing the user who owns the mobile terminal 40 under constant photographing conditions. From the photo creation game machine 10, the whole body image P111 extracted from the photographed image is supplied to the recommendation server 30 via the image management server 20. The photographed image may be directly supplied from the photo creation game machine 10 to the recommendation server 30, and the recommendation server 30 may extract the whole body image P111 from the photographed image.

The recommendation server 30 analyzes the whole body image P111. For example, a user's style can be analyzed by acquiring skeletal information using machine learning. Then, the recommendation server 30 generates recommendation information for recommending products and services to the user based on the obtained analysis results. For example, recommendation information R131 for recommending fashion-related products and services that match the user's style (height and body shape) is generated based on the analysis result of the full-body image P111. The recommendation server 30 transmits the generated recommendation information R131 to the mobile terminal 40 owned by the user.

However, the present invention is not limited to this, and the photo creation game machine 10 may analyze the whole body image P111 and supply the obtained analysis result to the recommendation server 30 via the image management server 20 together with the whole body image P111. In this case, the recommendation server 30 generates recommendation information R131 based on the analysis result from the photo creation game machine 10.

Further, the photo creation game machine 10 analyzes the whole body image P111, generates recommendation information R131 based on the obtained analysis result, and sends the generated recommendation information R131 to the image management server 20 along with the whole body image P111. The information may also be supplied to the recommendation server 30 via the service provider. In this case, the recommendation server 30 transmits the recommendation information R131 from the photo creation game machine 10 to the mobile terminal 40.

With the above-described flow, recommendation information based on the user's full-body image can be presented to the user on the mobile terminal 40.

(Presentation of recommendation information based on other information)
Recommendation information may be presented to the user based on information other than the captured image generated by the photo creation game machine 10.

For example, recommendation information for the user may be generated based on the captured image generated by the photo creation game machine 10 and the location information of the store or facility where the photo creation game machine 10 is installed. Recommendation information for the user may be generated based on the captured image generated by the device 10 and location information indicating the current location of the mobile terminal 40.

Further, recommendation information for the user may be generated based on the captured image generated by the photo creation game machine 10 and the user's membership information and profile information (age, gender, etc.) on an image management site or a recommendation site.

Furthermore, in the captured image generated by the photo creation game machine 10, the high school that the user attends is identified from the uniform worn by the user, and based on the identified high school, a university or vocational school is recommended to the user. Recommendation information representing the information may be generated.

(Utilization of user data)
The user data stored in the user DB 530 may not only be viewed on the above-mentioned my page, but may also be reflected in a profile on an SNS linked to an image acquisition site or a recommendation site.

(Application on mobile device)
In the above, it is assumed that the recommendation information is presented on the web page, but the recommendation information may be presented by a dedicated application (dedicated application) installed on the mobile terminal 40. In this case, the recommendation information may be generated by the recommendation server 30 that has analyzed the analysis image, as in the embodiment described above, or may be generated based on the analysis results from the recommendation server 30. May be generated by the app. In the latter case, the dedicated application of the mobile terminal 40 may further perform analysis on the analysis image.

(Other examples of recommendation information)
Recommendation information is not limited to information for recommending products and services related to makeup or fashion related products and services as described above. The recommendation information may be, for example, information for recommending to the user the type of plastic surgery, a destination for travel, a destination for further study, a place for employment, a place to move to, and the like.

The series of processes described above can be executed by hardware or software. When the series of processes described above is executed by software, the programs that make up the software are installed from a network or a recording medium.

For example, as shown in FIG. 3, this recording medium is a magnetic recording medium on which a program is recorded, which is distributed in order to distribute the program to the administrator of the recommendation server 30 (server 200), separately from the main body of the device. It is composed of a removable medium 119 such as a disk (including a flexible disk), an optical disk (including a CD-ROM and a DVD), a magneto-optical disk, or a semiconductor memory. In addition to these, the recording medium includes a ROM 212 that is preinstalled in the device main body and stores a program distributed to the administrator, a hard disk included in the storage section 218, and the like.

Note that in this specification, the step of writing a program to be recorded on a recording medium is not only a process that is performed chronologically in the stated order, but also a process that is not necessarily performed chronologically, but may be performed in parallel or in parallel. It also includes processes that are executed individually.

Further, the embodiments of the present technology are not limited to the embodiments described above, and various changes can be made without departing from the gist of the present technology.

Note that the effects described in this specification are merely examples and are not limited, and other effects may also exist.

1 Recommendation System, 10 Photo Creation Game Machine, 20 Image Management Server, 30 Recommendation Server, 40 Mobile Terminal, 131 Camera, 132 Touch Panel Monitor, 133 Lighting Device, 151 Printer, 510 Control Unit, 511 Photography Processing Unit, 512 Image Processing Unit , 513 print processing unit, 520 control unit, 521 web page provision unit, 522 image analysis unit, 523 recommendation information generation unit, 524 data management unit, 530 user DB

Claims (9)

a receiving unit that receives a photographed image in which a plurality of people are photographed from a photo creation game machine that generates a photographic image by photographing a plurality of people as subjects in a photographing space having a camera and a lighting device;
Receive selection information representing the user's own face image from among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user , and receive the selection information representing the face image of the user himself/herself from among the face images extracted from the photographed image linked to the user; a generation unit that generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing the face image . A photo creation game machine that generates a photographed image showing multiple subjects by photographing multiple people as subjects in a photography space having a camera and a lighting device;
Receive selection information representing the user's own face image from among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user , and receive the selection information representing the face image of the user himself/herself from among the face images extracted from the photographed image linked to the user; A recommendation system comprising: a server that generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing the face image . The photo creation game machine generates a processed image in which a subject area of the photographed image is subjected to predetermined image processing, and transmits it to the server together with the photographed image that has not been subjected to the image processing;
Among the images transmitted from the photo creation game machine, the server uses the photographed image that has not been subjected to the image processing as an analysis image, and performs analysis on the user's own face image extracted from the analysis image. The recommendation system according to claim 2, which performs analysis. The photo creation game machine generates the photographed image as the analysis image by presenting the user with instruction information instructing a standing position and a face direction in the photographing space. Recommendation system described. The server causes the mobile terminal to display a plurality of processed face images extracted from the photographed image subjected to the image processing, and selects one of the plurality of face images extracted from the analysis image on the mobile terminal. The recommendation system according to claim 4, wherein the face image corresponding to the processed face image selected by the user is analyzed at the terminal. The server generates the recommendation information based on an analysis result obtained by analyzing the face image and an answer selected by the user in response to a question to the user. The recommendation system described in any of the above. The recommendation system according to claim 6, wherein the recommendation information includes information for recommending at least one of the product and the service related to makeup. A photo creation game machine
By photographing multiple people as subjects in a photography space equipped with a camera and a lighting device, a photographed image showing multiple subjects is generated,
The server is
Receive selection information representing the user's own face image from among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user , and receive the selection information representing the face image of the user himself/herself from among the face images extracted from the photographed image linked to the user; A recommendation method that generates recommendation information for recommending at least one of a product and a service to the user based on an analysis result obtained by analyzing the face image . A photo creation game machine that generates a photographed image showing multiple subjects by photographing multiple people as subjects in a photography space having a camera and a light source;
Receive selection information representing the user 's own face image among the face images extracted from the photographed image linked to the user from a mobile terminal owned by the user, and receive the selection information representing the user's own face image from among the face images extracted from the photographed image linked to the user; a server that supplies recommendation information for recommending at least one of a product and a service to the user to the mobile terminal of the user based on an analysis result obtained by analyzing the face image; A recommendation system equipped with

Images