JP2023039542A - Server, recommendation system, and recommendation method - Google Patents

Abstract

To make better recommendations.SOLUTION: A photo-creation game machine generates a photographic image by photographing a user as a subject in a photographing space including a camera and a lighting device. A server generates recommendation information for recommending at least either a product or a service to the user based on analysis results obtained by analyzing a subject area of the photographic image. The present technique can be applied to a recommendation system that presents a recommended makeup to each user.SELECTED DRAWING: Figure 7

Description

The present technology relates to a recommendation system and a recommendation method, and more particularly to a recommendation system and a recommendation method that enable more appropriate recommendations.

In recent years, many systems have been provided that use a user's face image or answer a question to process the face image so that the user's desired makeup is applied or output advice. there is

In Patent Document 1, a database in which cosmetic color information and product information are registered is searched for cosmetic product color information and product information that are optimal for the skin color of a face region in a user image captured by a user terminal. A makeup advice device for transmitting to a user terminal is disclosed.

Japanese Unexamined Patent Application Publication No. 2010-211308

As in the system disclosed in Japanese Patent Laid-Open No. 2002-200310, there is a risk that appropriate recommendations cannot be made with face images captured by users using their own mobile terminals.

The present technology has been made in view of such circumstances, and enables more appropriate recommendations to be made.

A recommendation system according to a first aspect of the present technology includes a photograph creation game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a lighting device, and an analysis of a subject area of the photographed image. and a server that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by performing the recommendation system.

In a recommendation method according to a first aspect of the present technology, a photo-creating game machine photographs a user as a subject in a photographing space having a camera and a lighting device to generate a photographed image; The recommendation method generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by analyzing areas.

A recommendation system according to a second aspect of the present technology includes a photograph creation game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a light source, and an analysis of a subject area of the photographed image. a recommendation server that supplies recommendation information for recommending at least one of products and services to the user to the mobile terminal owned by the user, the recommendation information being generated based on the analysis result obtained by the system.

In the present technology, a photographed image is generated by photographing a user as a subject in a photographing space having a camera and a lighting device, and the subject area of the photographed image is analyzed based on the analysis results obtained. and generating recommendation information for recommending products and/or services to the user, or supplying the recommendation information to a mobile terminal owned by the user.

According to the present technology, it is possible to make more appropriate recommendations.

Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure which shows the structural example of a recommendation system. It is a block diagram showing a hardware configuration example of a photo creation game machine. It is a block diagram which shows the hardware configuration example of a server. It is a block diagram which shows the hardware structural example of a portable terminal. FIG. 10 is a diagram showing an example of the internal configuration of a shooting space of the photo creation game machine; FIG. 10 is a diagram illustrating the flow of presentation of recommended information based on face images; It is a block diagram which shows the functional structural example of a recommendation system. FIG. 4 is a diagram showing an example of user data; It is a figure explaining the flow of operation|movement of a recommendation system. FIG. 11 is a diagram showing an example of a live view screen during face analysis photographing; FIG. 10 is a diagram showing an example of a live view screen during normal shooting; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram showing an example of a recommended site screen displayed on a mobile terminal; FIG. 10 is a diagram illustrating the flow of presentation of recommendation information based on a whole-body image;

Below, the form (henceforth embodiment) for implementing this technique is demonstrated. The explanation is given in the following order.

1. Configuration of recommendation system 2 . 2. Shooting in the shooting space of the photo-creating game machine; Presentation of recommended information based on facial images4. Modification

<1. Configuration of recommendation system>
(Outline of recommendation system)
FIG. 1 is a diagram showing an overview of a recommendation system according to an embodiment of the present technology.

The recommendation system 1 shown in FIG. 1 is composed of a photo creation game machine 10, an image management server 20, a recommendation server 30, and mobile terminals 40-1 and 40-2.

Each device constituting the recommendation system 1 is directly or indirectly connected to the network NW and communicates with each other. The network NW includes the Internet and local area networks, and may be composed of wired communication lines or wireless communication lines.

Also, in the recommendation system 1, the image management server 20 and the recommendation server 30 are configured separately, but they may be integrated.

The photo-creating game machine 10 is installed in a store such as a game center or an amusement facility, and provides a photo-creating game to the user by receiving coins for the play fee. Specifically, the photo-creating game machine 10 accepts an editing operation (scribbling) on a photographed image obtained by photographing the user, and provides the photographed image to the user by printing it on sticker paper.

Users of the photo creation game machine 10 are mainly high school girls and young women. In the photo-creating game machine 10, mainly two users can enjoy the game per set. Of course, in the photo creation game machine 10, three or more users or one user can enjoy the game.

Although only one photo-creating game machine 10 is shown in FIG. 1, the recommendation system 1 actually includes a plurality of photo-creating game machines 10 .

The image management server 20 manages captured images obtained by shooting with the photo creation game machine 10, and is a member of a website (image acquisition site) operated by a manufacturer that manufactures and sells the photo creation game machine 10. Manage user information.

A user who has become a member can browse the photographed images obtained by photographing with the photograph creation game machine 10 on the portable terminal 40-1 or the portable terminal 40-2.

The image management server 20 acquires, for each user who has become a member, a photographed image obtained by photographing with the photographing game machine 10 from the photographing game machine 10 (such as a store where the photographing game machine 10 is installed). do. A captured image for each user can be supplied to the recommendation server 30 as needed.

The recommendation server 30 provides a website (recommendation site) that recommends products and services to each user, using the photographed image of each user from the image management server 20 . Specifically, the recommendation server 30 generates recommendation information for recommending products and services based on analysis results obtained by analyzing captured images of each user, and generates recommendation information for recommending products and services. 40-1 and portable terminal 40-2.

The mobile terminals 40-1 and 40-2 are composed of mobile phones (smartphones), mobile PCs (Personal Computers), tablet terminals, etc. owned by respective users. Hereinafter, the mobile terminals 40-1 and 40-2 are simply referred to as mobile terminals 40 when not distinguished from each other. Although only two mobile terminals 40 are shown in FIG. 1, a plurality of mobile terminals 40 actually exist in the recommendation system 1 .

(Hardware configuration of photo creation game machine)
FIG. 2 is a block diagram showing a hardware configuration example of the photo creation game machine 10. As shown in FIG.

As shown in FIG. 2, the photo creation game machine 10 has a pre-selection unit 102, a photographing unit 103, an editing unit 104, and a printing unit 105 connected to a PC unit 101 via an external input/output interface 106. It is composed by

A CPU (Central Processing Unit) 111 , a ROM (Read Only Memory) 112 , and a RAM (Random Access Memory) 113 constituting the PC section 101 are interconnected by a bus 114 .

The CPU 111 executes a predetermined program and controls the overall operation of the photo-creating game machine 10 . ROM 112 stores programs and data executed by CPU 111 . The RAM 113 temporarily stores data and programs processed by the CPU 111 .

An input/output interface 115 is further connected to the bus 114 . A storage unit 116 , a communication unit 117 and a drive 118 are connected to the input/output interface 115 .

Storage unit 116 is a non-volatile storage medium such as a hard disk or flash memory. The storage unit 116 stores various setting information and the like supplied from the CPU 111 . Information stored in storage unit 116 is read by CPU 111 as appropriate.

A communication unit 117 is an interface of a network such as the Internet. A communication unit 117 communicates with an external device under the control of the CPU 111 . Communication unit 117 transmits the image selected by the user to image management server 20 . The image transmitted from the communication unit 117 is stored in a predetermined storage area, displayed on the mobile terminal 40 accessing the image acquisition site, or downloaded to the mobile terminal 40 .

A removable medium 119 such as an optical disk or a semiconductor memory is appropriately mounted in the drive 118 . Programs and data read from the removable medium 119 by the drive 118 are supplied to the CPU 111 and stored in the storage unit 116 or installed.

The external input/output interface 106 is connected to the input/output interface 115 . Each unit is controlled by the PC unit 101 via the external input/output interface 106 .

The pre-selection unit 102 performs pre-selection processing for the user before shooting. The pre-selection unit 102 is composed of a touch panel monitor 121 , a speaker 122 and a coin processing unit 123 .

The touch panel monitor 121 displays various selection screens under the control of the PC unit 101 and accepts user operations on the selection screens. An input signal representing the content of the user's operation is supplied to the PC unit 101, and various settings are performed.

The speaker 122 outputs guidance voices for operations on various selection screens, background music (BGM), sound effects, and the like.

The coin processing unit 123 detects insertion of coins into the coin slot. When coin processing unit 123 detects that a predetermined amount of coins has been inserted, coin processing unit 123 outputs an activation signal instructing the start of a game to PC unit 101 .

The photographing unit 103 performs photographing processing for the user in the photographing space formed within the housing of the photograph creation game machine 10 . The imaging unit 103 is composed of a camera 131 , a touch panel monitor 132 , an illumination device 133 and a speaker 134 .

The camera 131 performs photographing according to shutter control by the PC section 101 and outputs image data obtained by photographing to the PC section 101 .

A touch panel monitor 132 is provided near the camera 131 . The touch panel monitor 132 has a function as a live view monitor that displays moving images captured by the camera 131 and a function that displays various GUIs and receives user selection operations.

The lighting device 133 is a strobe unit provided in the shooting space and emits light according to the lighting control signal supplied from the PC section 101 .

A speaker 134 is provided at a predetermined position in the shooting space, and outputs guide voices, BGM, sound effects, and the like for shooting processing.

The editing unit 104 performs editing processing after photographing. The editing unit 104 is composed of a tablet built-in monitor 141 , touch pens 142 A and 142 B, and a speaker 143 .

The tablet built-in monitor 141 and the touch pens 142A and 142B are provided on each of the two surfaces of the housing that constitutes the editing unit 104 . The touch pen 142A and the touch pen 142B are simply referred to as the touch pen 142 when there is no need to distinguish between them.

The tablet built-in monitor 141 displays an edit screen under the control of the PC unit 101 and accepts a user's operation using the touch pen 142 on the edit screen. An input signal representing the content of the user's operation is supplied to the PC unit 101, and the captured image to be edited is edited.

The speaker 143 is built in a part of the housing that constitutes the editing unit 104, and outputs guide voices, BGM, sound effects, etc. for the editing process.

Printing unit 105 is configured to include printer 151 . A sticker paper unit 152 is attached to the printer 151 .

The printer 151 prints an image on the sticker paper contained in the sticker paper unit 152 based on the print data supplied from the PC section 101 . The printer 151 discharges the seal paper on which the image is printed to the seal paper discharge port.

(Hardware configuration of image management server/recommendation server)
FIG. 3 is a block diagram showing a hardware configuration example of the server 200 that constitutes each of the image management server 20 and the recommendation server 30. As shown in FIG.

In FIG. 3, CPU 211 executes various processes according to programs stored in ROM 212 or programs loaded in RAM 213 . The RAM 213 also appropriately stores data necessary for the CPU 211 to execute various processes.

CPU 211 , ROM 212 and RAM 213 are interconnected via bus 214 . An input/output interface 215 is also connected to this bus 214 .

An input unit 216 , an output unit 217 , a storage unit 218 and a communication unit 219 are connected to the input/output interface 215 .

The input unit 216 is composed of a keyboard, a mouse, and the like. The output unit 217 includes a display made of liquid crystal, organic EL (Electro-Luminescence), etc., a speaker, and the like. Storage unit 218 is configured by a hard disk or the like. The communication unit 219 includes a wired communication module for wired communication, a wireless communication module for wireless communication, and the like.

The communication unit 219 performs communication processing via the network NW.

A drive 220 is also connected to the input/output interface 215 as necessary, and a removable medium 221 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted as appropriate. Computer programs read from them are installed in the storage unit 218 as necessary.

(Hardware configuration of mobile terminal)
FIG. 4 illustrates an example hardware configuration of the mobile terminal 40 .

In FIG. 4, CPU 411 executes various processes according to programs stored in ROM 412 or programs loaded in RAM 413 . The RAM 413 also appropriately stores data necessary for the CPU 411 to execute various processes.

CPU 411 , ROM 412 and RAM 413 are interconnected via bus 414 . An input/output interface 415 is also connected to this bus 414 .

An input unit 416 , an output unit 417 , a storage unit 418 and a communication unit 419 are connected to the input/output interface 415 .

The input unit 416 includes keys, buttons, a touch panel, a camera, a microphone, and the like. The output unit 417 is composed of a display formed of a liquid crystal, an organic EL, or the like together with a touch panel, a speaker, and the like. The storage unit 418 is configured by, for example, a NAND flash memory. The communication unit 419 is configured by a communication module or the like that performs wireless communication.

<2. Shooting in the Shooting Space of the Photo Creation Game Machine>
Here, photographing in the photographing space of the photograph creating game machine 10 will be described.

FIG. 5 is a diagram showing an example of the internal configuration of a photography space formed within the housing of the photography game machine 10. As shown in FIG.

In the shooting space, the user faces the camera unit CU in which the camera 131 and the touch panel monitor 132 are built in, and faces shooting. On the front side of the user in the photographing space (on the left side in FIG. 5), in addition to the camera unit CU described above, there are an upper strobe unit 133-1 above the camera unit CU and a foot strobe unit below the camera unit CU as lighting devices 133. 133-2 are provided.

Further, left and right strobe units 133-3 are provided on the upper left and right sides of the user facing the camera unit CU in the shooting space.

Furthermore, although not shown, in the shooting space, a back strobe unit may be provided above the back of the user facing the camera unit CU, or a ceiling strobe unit may be provided on the ceiling of the shooting space.

Each strobe unit (illumination device 133 ) provided in the photographing space in this manner has a strobe light emitting tube therein and emits strobe light at the photographing timing of the camera 131 .

Specifically, the upper strobe unit 133-1 irradiates the strobe light L1 centering on the face from diagonally above the front of the user at the photographing timing of the camera 131. FIG. The foot strobe unit 133-2 emits strobe light L2 from the front and obliquely below the user at the photographing timing of the camera 131, focusing on the lower half of the body from the feet. The left and right strobe units 133-3 irradiate strobe light L3 from right and left obliquely above the user centering on the left and right temporal regions at the photographing timing of the camera 131. FIG.

Further, inside the upper strobe unit 133-1, the foot strobe unit 133-2, and the ceiling strobe unit (not shown), in addition to the strobe arc tube, there are constant light sources such as LED (Light Emitting Diode) bulbs. be provided. As a result, the brightness in the shooting space can be kept constant even at times other than the shooting timing.

That is, in the shooting space of the photo-creating game machine 10, shooting is performed in an optical environment such as a constant amount of light and a constant light emission time.

A live view screen including a live view image captured by the camera 131 is displayed on the touch panel monitor 132 before the shooting timing of the camera 131 . On the live view screen, the user is presented with instruction information that instructs the user on the standing position and the orientation of the face.

That is, in the photographing space of the photo-creating game machine 10, photographing is performed with the size and orientation of the face fixed with respect to the angle of view.

Furthermore, the camera 131 is adjusted in advance in terms of white balance, brightness, hue, angle of view, etc. for each photo creation game machine 10 in order to suppress variations in hue, brightness, angle of view, and the like. These adjustments are made so that the values of each parameter fall within a certain range.

That is, in the photographing space of the photograph making game machine 10, photographing is performed in a state in which the individual difference of the camera 131 for each photograph making game machine 10 is suppressed as much as possible.

As described above, in the shooting space of the photo-creating game machine 10, shooting is performed under constant shooting conditions such as the optical environment, the size and orientation of the face relative to the angle of view, and individual differences in the camera 131. It is possible to stabilize the appearance of the photographed image obtained by the photographing (especially the appearance of the user's face area) without fluctuation.

Note that the shooting of the camera 131, the display of the touch panel monitor 132, and the light emission of each strobe unit (lighting device 133) are all controlled by the CPU 111 (FIG. 2).

<3. Presentation of Recommended Information Based on Face Image>
An example in which recommendation information is presented to a user based on a face image extracted from a stable photographed image will be described below.

(Flow of presentation of recommended information)
FIG. 6 is a diagram illustrating the flow of presentation of recommendation information based on face images.

FIG. 6 shows a photo-creating game machine 10, a recommendation server 30, and a mobile terminal 40 that constitute the recommendation system 1. As shown in FIG.

As described above, in the photo-creating game machine 10, the user who owns the portable terminal 40 is taken as a subject under fixed shooting conditions, so that a stable shot image can be obtained. The face image P<b>11 extracted from the photographed image is supplied from the photo creation game machine 10 to the recommendation server 30 via the image management server 20 . The photographed image may be directly supplied from the photo creation game machine 10 to the recommendation server 30, and the face image P11 may be extracted from the photographed image in the recommendation server 30. FIG.

The recommendation server 30 analyzes the face image P11. Based on the obtained analysis results, the recommendation server 30 generates recommendation information for recommending products and services to the user. For example, recommendation information R31 for recommending makeup and cosmetics that match the user's face is generated based on the analysis result of the face image. The generated recommendation information R31 is transmitted from the recommendation server 30 to the mobile terminal 40 owned by the user.

Not limited to this, the photo creation game machine 10 may analyze the face image P11 and supply the obtained analysis result to the recommendation server 30 via the image management server 20 together with the face image P11. In this case, the recommendation server 30 generates recommendation information R31 based on the analysis result from the photo creation game machine 10. FIG.

Further, the photo creation game machine 10 analyzes the face image P11, generates recommendation information R31 based on the obtained analysis result, and sends the generated recommendation information R31 to the image management server 20 together with the face image P11. may be supplied to the recommendation server 30 via the In this case, the recommendation server 30 transmits the recommendation information R31 from the photo creation game machine 10 to the mobile terminal 40 .

With the flow as described above, it is possible to present recommended information based on the user's face image to the user on the portable terminal 40 .

(Function configuration example of recommendation system)
FIG. 7 is a block diagram showing a functional configuration example of the recommendation system 1 that realizes presentation of recommendation information based on the face image described above.

FIG. 7 shows the photo creation game machine 10, the image management server 20, the recommendation server 30, and the portable terminal 40 that constitute the recommendation system 1. Detailed functional configurations of the image management server 20 and the portable terminal 40 are shown in FIG. is omitted.

The photo-making game machine 10 includes a control section 510 in addition to the camera 131, the touch panel monitor 132, the illumination device 133, and the printer 151 described above. Each functional block constituting control unit 510 is implemented by CPU 111 executing a predetermined program.

The control unit 510 is configured to include a shooting processing unit 511 , an image processing unit 512 and a print processing unit 513 .

The imaging processing unit 511 executes imaging processing for the user by controlling the camera 131, the touch panel monitor 132, and the lighting device 133 provided in the imaging space.

An image processing unit 512 performs various types of image processing on a plurality of captured images.

For example, the image processing unit 512 performs predetermined image processing on the subject area of the captured image. Specifically, the image processing unit 512 processes (enlarges/reduces, transforms, changes color, etc.) or modifies the outline of the user's face, eyes, nose, mouth, cheeks, and other face parts appearing in the captured image. Apply retouching processing. The image processing unit 512 also performs editing processing on the captured image in accordance with the user's editing operation (doodle).

Editing processing is applied to a photographed image selected by the user according to the user's editing operation, while retouching processing is basically performed uniformly on all photographed images regardless of the user's operation. applied to By performing such image processing, a processed image that has been subjected to image processing is generated separately from a photographed image that has not been subjected to image processing.

The print processing unit 513 controls the printer 151 to print the post-processing image on a sticker paper and output the image.

In addition, the post-processing image is transmitted from the photo creation game machine 10 to the image management server 20 together with the photographed image that has not undergone image processing, and is managed for each user.

The recommendation server 30 has a control unit 520 and a user DB 530 . Each functional block constituting the control unit 520 and the user DB 530 are implemented by the CPU 211 executing a predetermined program.

The control unit 520 is configured to include a web page providing unit 521 , an image analysis unit 522 , a recommendation information generation unit 523 and a data management unit 524 .

The web page providing unit 521 creates a web page that constitutes a website (recommended site) viewed on the mobile terminal 40 and provides the web page to the mobile terminal 40 .

When a photographed image of the user corresponding to the portable terminal 40 that has accessed the recommended site is acquired from the image management server 20, the image analysis unit 522 analyzes the subject area of the photographed image. The analysis by the image analysis unit 522 is performed with a photographed image (original image) that has not been subjected to image processing among the photographed images managed by the image management server 20 as an analysis object. Specifically, the image analysis unit 522 analyzes the face image extracted from the captured image to be analyzed. Hereinafter, a captured image to be analyzed (a captured image that has not undergone image processing) will be referred to as an analysis image.

The recommendation information generation unit 523 generates recommendation information for the user corresponding to the face image based on the analysis result obtained by the analysis of the image for analysis (face image) by the image analysis unit 522 .

The data management unit 524 manages user data of users registered in the recommended site among the users corresponding to the portable terminals 40 that have accessed the recommended site. User data is stored in the user DB 530 .

FIG. 8 is a diagram showing an example of user data.

As shown in FIG. 8, the user data d530 stored in the user DB 530 is configured by associating a user ID with a face image ID and one or more pieces of past recommendation information.

The user ID is user-specific identification information registered in the recommendation site. The face image ID is identification information specific to the face image analyzed for the user identified by the user ID. The past recommendation information is recommendation information generated for the user identified by the user ID. The past recommendation information is added and accumulated, for example, each time the face image specified by the face image ID is analyzed.

With the user data d530 having such a structure, a user registered with the recommendation site can view recommended information presented in the past.

(Operation of recommendation system)
The operation flow of the recommendation system will be described with reference to FIG. The process of FIG. 9 is started when the photo creation game machine 10 starts providing the game to the user.

First, in step S11, the photo-creating game machine 10 (photographing processing unit 511) executes photographing processing. In the photographing process, a plurality of photographed images are generated by photographing the user as a subject a plurality of times.

Specifically, in the photographing process, for example, the first photographing of a plurality of times of photographing is performed as face analysis photographing for generating an analysis image, and the second and subsequent photographings are performed as images. This is performed as normal photography for generating a photographed image to be processed.

FIG. 10 is a diagram showing an example of a live view screen displayed on the touch panel monitor 132 during photographing for face analysis.

In the live view screen shown in FIG. 10, substantially square live view display areas 611A and 611B are arranged side by side. Below the live-view display areas 611A and 611B, text indicating that shooting for face analysis is to be performed as instruction information for instructing the standing position and face orientation, and a frame (live-view display areas 611A and 611B). A text prompting you to put your face in the front and shoot is displayed.

The two main users of the photo creation game machine 10 adjust their standing positions and face orientations so that their faces fit within the live view display areas 611A and 611B.

When the photographing timing comes in such a state, photographing for face analysis is performed and an image for analysis is generated.

On the other hand, FIG. 11 is a diagram showing an example of a live view screen displayed on the touch panel monitor 132 during normal shooting.

In the live view screen shown in FIG. 11, a vertically long rectangular live view display area 621 is provided in the center of the screen. On the left and right sides of the live view display area 621, sample pose images in which a model in a sample pose is photographed as a subject are displayed every multiple times (for example, five times) of normal shooting.

The two main users of the photo creation game machine 10 adjust their own poses and facial expressions while checking the sample pose images displayed on the left and right sides of the live view display area 621 and the live view display area 621 . conduct.

When it is time to shoot in such a state, normal shooting is performed and a shot image is generated.

Below the live view display area 621, a shooting result display area 622 is provided in which a shot image generated by normal shooting is displayed. In the example of FIG. 11, five shot images are displayed as the shooting results of five normal shots.

As described above, when one image for analysis and five shot images are generated in the shooting process, the process proceeds to step S12, and the image processing unit 512 performs image processing on the shot images.

As described above, for example, the five captured images are subjected to retouching processing regardless of the user's operation, and are subjected to editing processing according to the user's editing operation. As a result, five processed images to which image processing has been performed are generated. Here, it is assumed that the analysis image is also subjected to retouching processing. Hereinafter, the image for analysis that has undergone retouching is referred to as a post-processing image for analysis.

In step S<b>13 , the print processing unit 513 controls the printer 151 to print at least one of the five post-processing images on a sticker paper and output the printed image.

After that, in step S 14 , the communication unit 117 ( FIG. 2 ) of the photo-making game machine 10 transmits the five processed images and the analysis images (including the processed analysis images) to the image management server 20 .

Note that a captured image (original image) that has not been subjected to image processing and is generated by normal capturing may be further transmitted to the image management server 20 . In this case, instead of using the original image generated by face analysis photography as the image for analysis, among the original images generated by normal photography, the original image capable of face recognition required for face image analysis is analyzed. Alternatively, the image may be transmitted to the image management server 20 as an image for use.

The processed image transmitted to the image management server 20 is displayed on the mobile terminal 40 of the user who has become a member of the image acquisition site, or is downloaded to the mobile terminal 40 .

Now, in step S<b>21 , the communication unit 219 ( FIG. 3 ) of the image management server 20 transmits analysis images (including processed analysis images) to the recommendation server 30 .

Here, it is assumed that members registered in the image acquisition site provided by the image management server 20 and members registered in the recommended site provided by the recommendation server 30 are linked by a predetermined method. For example, the user can register himself/herself as the same member in each of the image acquisition site and the recommendation site by using the e-mail address of the mobile terminal 40 or the SNS (Social Networking Service) account as a common user ID. .

Note that the timing at which the analysis image is transmitted from the image management server 20 to the recommendation server 30 may not be every time the user plays the game, but rather every relatively long period of time, such as once a month. . This is because the user's face appearing in the analysis image does not change much over time compared to the user's face appearing in the post-processing image that undergoes various types of image processing.

In such a state, for example, when the user performs an operation on the login screen displayed by accessing the recommended site from the mobile terminal 40, in step S41, the communication unit 419 of the mobile terminal 40 accesses the recommended site. A login request for logging in is transmitted to the recommendation server 30 .

In step S31, the web page providing unit 521 of the recommendation server 30 accepts login in response to a login request from the mobile terminal 40, and displays a selection screen on the mobile terminal 40 for allowing the user to select himself/herself from the subjects appearing in the analysis images. to display.

Since an analysis image often includes a plurality of subjects, such as two persons, a single analysis image can be associated with a plurality of users (members). In this case, even if it is possible to specify an analysis image linked to a certain user, it is not possible to specify which subject is the user in the analysis image.

Therefore, by allowing the user to select himself/herself from the subjects appearing in the analysis image, it is possible to specify which subject is the user in the analysis image. Specifically, a plurality of face images extracted from the image for analysis are displayed on the selection screen, and the user is allowed to select one of the face images. In the following, it is assumed that two users appear in the analysis image and two face images are displayed on the selection screen.

In step S<b>42 , the communication unit 419 of the mobile terminal 40 transmits selection information representing the selection content (selected face image) to the recommendation server 30 in accordance with the user's operation on the selection screen.

In step S32, the image analysis unit 522 of the recommendation server 30 identifies which subject in the analysis image is the user of the mobile terminal 40 based on the selection information from the mobile terminal 40, and analyzes the subject area. conduct. Specifically, among a plurality of face images extracted from the analysis image, the face image represented by the selection information is analyzed.

Next, in step S<b>33 , the web page providing unit 521 of the recommendation server 30 causes the mobile terminal 40 to display a question screen for selecting a question for the user of the mobile terminal 40 and an answer to the question.

In step S<b>43 , the communication unit 419 of the mobile terminal 40 transmits answer information indicating the selected content of the answer to the question to the recommendation server 30 in accordance with the user's operation on the question screen. The questions to the user include, for example, questions about the user's skin color, questions about the makeup that the user usually wears, and the like.

Display of such a question screen and transmission of answer information are often performed multiple times.

After that, in step S34, the recommendation information generation unit 523 of the recommendation server 30 uses the analysis result obtained by the analysis of the image for analysis by the image analysis unit 522 and a plurality of pieces of answer information from the mobile terminal 40. Based on this, recommendation information for the user of the mobile terminal 40 is generated. Here, for example, recommendation information for recommending at least one of makeup-related products and services is generated based on the analysis result of the face image and the answer information to the above-described question.

In step S<b>35 , the web page providing unit 521 of the recommendation server 30 transmits the web page including the recommendation information to the mobile terminal 40 .

Then, in step S44, the output unit 417 (FIG. 4) of the mobile terminal 40 presents the user with a Web page including the recommendation information from the recommendation server 30. FIG.

(Screen transition of recommended site)
Next, in the operation flow of the recommendation system 1 described above, screen transitions of recommended sites browsed by the user using the portable terminal 40 will be described. Here, an example of screen transition from the time a user registered as a member of a recommended site logs in to the recommended site to the presentation of recommended information is shown.

First, when the mobile terminal 40 accesses the recommended site, the login screen 710 shown in FIG. 12 is displayed on the display (output unit 417) of the mobile terminal 40. FIG. A login screen 710 is provided with a user icon display area 711 , a password input box 712 , and a login button 713 .

The user icon display area 711 is an area where an icon set for each user of the mobile terminal 40 is displayed. The icon displayed in the user icon display area 711 may be a human figure automatically set by the system as shown in FIG. 12, or may be a user's face image set by the user. .

A password entry box 712 is a text box for the user to enter a password for logging into the recommended site. When the login button 713 is pressed by the user with the correct password entered in the password entry box 712, a login request is transmitted from the portable terminal 40 to the recommendation server 30 (step S41 in FIG. 9), and the login is completed. .

When the login is completed, the selection screen 720 shown in FIG. 13 is displayed on the display of the mobile terminal 40 . The selection screen 720 is provided with face image display areas 721A and 721B, selection buttons 722A and 722B, and a diagnosis start button 723 .

The face image display areas 721A and 721B are areas where the face images of the two users extracted from the analysis image associated with the user of the mobile terminal 40 are displayed. The face images displayed in the face image display areas 721A and 721B are the face images extracted from the post-processing analysis image. That is, in the face image display areas 721A and 721B, a face image that has undergone retouch processing for processing or modifying the outline of the face, eyes, nose, mouth, cheeks, etc. is displayed.

Note that these face images may be extracted from the post-processing analysis image in the recommendation server 30, or may be extracted from the post-processing analysis image in the photo creation game machine 10 and then transmitted to the recommendation server 30. good.

The selection buttons 722A and 722B are buttons for the user to select his/her own face from the face images displayed in the face image display areas 721A and 721B. When the selection button 722A is pressed, the face image displayed in the face image display area 721A is selected, and when the selection button 722B is pressed, the face image displayed in the face image display area 721B is selected.

When the diagnosis start button 723 is pressed by the user while one of the selection buttons 722A and 722B is selected, selection information representing the selected face image is transmitted from the portable terminal 40 to the recommendation server 30 (FIG. 9 step S42).

The recommendation server 30 starts analyzing the face image represented by the selection information (step S32 in FIG. 9). Specifically, the feature points of the face in the face image are detected, and the face is mapped on a preset feature quantity space. A face type is set in each area of the feature amount space, and the face type of the user is specified by mapping the face on the feature amount space.

Here, the face image to be analyzed is not the post-processing analysis image, but the face image extracted from the analysis image (original image) that has not undergone image processing. Since the face image extracted from the post-processing image for analysis is deformed by retouching, etc., detection of feature points is not performed correctly, and analysis that is originally required is not performed. On the other hand, the face image extracted from the image for analysis is not subjected to deformation by retouching, etc., and detection of feature points is performed correctly, so it is possible to perform the originally required analysis.

Now, when the diagnosis start button 723 is pressed, a diagnosis screen 730 shown in FIG. 14 is displayed on the display of the portable terminal 40 as one of the question screens described above. The diagnosis screen 730 is a question screen for asking the user about the tendency of the skin color and making the user answer. Diagnosis screen 730 is provided with answer buttons 731A and 731B and next button 732 .

The answer button 731A is a button for selecting that the skin color is more bluish as an answer to the question. The answer button 731B is a button for selecting that the skin color is more yellowish as an answer to the question.

When the user presses the next button 732 while either of the answer buttons 731A and 731B is selected, answer information representing the selected answer is transmitted from the portable terminal 40 to the recommendation server 30 (FIG. 9). step S43).

When the next button 732 is pressed, the display of the portable terminal 40 displays a diagnosis screen 740 shown in FIG. 15 as another one of the question screens described above. The diagnosis screen 740 is a question screen for asking the user whether or not to apply cheeks, and having the user answer. The diagnosis screen 740 is provided with answer buttons 741A and 741B and a next button 742 .

The answer button 741A is a button for selecting ticking as an answer to the question. The answer button 741B is a button for selecting not to blush as an answer to the question.

When the user presses the next button 742 while one of the answer buttons 741A and 741B is selected, answer information representing the selected answer is transmitted from the mobile terminal 40 to the recommendation server 30 (FIG. 9). step S43).

In the recommendation server 30, recommendation information for the user of the mobile terminal 40 is generated based on the facial image analysis result and a plurality of pieces of answer information from the mobile terminal 40 (step S34 in FIG. 9).

After that, a diagnosis result screen 750 shown in FIG. 16 is displayed on the display of the mobile terminal 40 . The diagnosis result screen 750 is a screen for presenting the preliminary recommendation information generated by the recommendation server 30 before generating the recommendation information. The preliminary recommendation information referred to here is information representing features and characteristics for deriving the product or service represented by the recommendation information. The diagnosis result screen 750 is provided with a recommended preliminary information display area 751 and an item button 752 .

The recommended preliminary information display area 751 is an area in which the above-described recommended preliminary information is displayed. In the example of FIG. 16, text and an image representing the lip color (honey terracotta in the figure) to be recommended to the user are displayed as preliminary recommendation information.

The item button 752 is a button for presenting recommended information (products and services) derived based on the preliminary recommended information displayed in the preliminary recommended information display area 751 .

That is, when item button 752 is pressed, diagnosis result screen 760 shown in FIG. 17 is displayed on the display of portable terminal 40 . The diagnosis result screen 760 is a screen for presenting recommendation information generated by the recommendation server 30 . A diagnosis result screen 760 is provided with a recommended information display area 761 and evaluation buttons 762A, 762B, and 762C.

The recommended information display area 761 is an area where recommended information is displayed. In the example of FIG. 17, as the recommendation information, words and images representing the lip (product) corresponding to the color presented as the preliminary recommendation information are displayed.

By tapping the recommended information display area 761 to make a selection, the user can access the product purchase screen displayed in the recommended information display area 761 and purchase the product.

The evaluation buttons 762A, 762B, and 762C are buttons for the user to evaluate the products and services presented as the recommended information (select preferences for the products and services). Specifically, the evaluation button 762A is a button for highly evaluating the product presented as the recommended information. The evaluation button 762B is a button for moderately evaluating a product presented as recommended information. The evaluation button 762C is a button for giving a low evaluation to the product presented as recommended information.

The recommendation information presented on the diagnosis result screen in this way is associated with the user ID as past recommendation information together with evaluation information representing the user's evaluation of the product or service presented as the recommendation information and preliminary recommendation information. , is stored in the user DB 530 as user data d530.

Also, a user who has logged in to the recommendation site from the mobile terminal 40 can check recommendation information (past recommendation information) that has been presented on the diagnosis result screen in the past by accessing My Page, which is a user-dedicated management page. can.

On My Page, in addition to viewing past recommended information, users can edit profile information such as the user's name, icon, and password on the recommended site, and check the history of purchased products. My page 770 shown in FIG. 18 is an example of my page when viewing past recommended information.

A diagnosis result list display area 771 and a past recommended information display area 772 are displayed on my page 770 in FIG. 18 .

The diagnosis result list display area 771 is an area in which recommended preliminary information presented most recently as a diagnosis result and answer information indicating answers to questions selected by the user are displayed. In the example of FIG. 18, text and an image representing the lip color recommended to the user are displayed as the preliminary recommendation information, and text and an image representing the skin color of the user's face are displayed as the answer information.

The past recommended information display area 772 is an area in which recommended information (products and services) presented in the past is displayed. In the example of FIG. 18, a plurality of lips (products) recommended to the user in the past are displayed as the recommendation information.

As described above, a photographed image is generated by photographing under fixed photographing conditions in the photograph creation game machine 10, and recommendation information is generated based on the analysis result of the face image extracted from the photographed image. be. In other words, unlike shooting using a user's mobile terminal, the amount of light fluctuates depending on the shooting conditions such as the optical environment, and the shot of the face image is not stable. Since the recommendation information is generated by the user, it is possible to make a more appropriate recommendation to the user.

In the above example, the recommended information is presented when the user logs into the recommendation site. By searching for a service, corresponding recommendation information may be presented.

<4. Variation>
Modified examples of the recommendation system to which the present technology is applied will be described below.

(Presentation of recommended information based on whole-body image)
An example of presenting recommended information to a user based on a face image extracted from a stable photographed image has been described above.

Without being limited to this, in the recommendation system to which the present technology is applied, it is also possible to present recommendation information to the user based on a full-body image of a person (whole-body image) extracted from a stable photographed image.

FIG. 19 is a diagram explaining the flow of presentation of recommendation information based on a whole-body image.

FIG. 19 shows a photo creation game machine 10, a recommendation server 30, and a mobile terminal 40 that constitute the recommendation system 1. As shown in FIG.

As described above, in the photo-creating game machine 10, the user who owns the portable terminal 40 is taken as a subject under fixed shooting conditions, so that a stable shot image can be obtained. A full body image P111 extracted from the photographed image is supplied from the photo creation game machine 10 to the recommendation server 30 via the image management server 20 . The photographed image may be directly supplied from the photo creation game machine 10 to the recommendation server 30, and the recommendation server 30 may extract the whole body image P111 from the photographed image.

The recommendation server 30 analyzes the whole-body image P111. For example, the user's style is analyzed by acquiring skeleton information using machine learning. Based on the obtained analysis results, the recommendation server 30 generates recommendation information for recommending products and services to the user. For example, recommendation information R131 for recommending fashion-related products and services that match the user's style (height and body shape) is generated based on the analysis result of the whole-body image P111. The generated recommendation information R131 is transmitted from the recommendation server 30 to the mobile terminal 40 owned by the user.

Not limited to this, the photo creation game machine 10 may analyze the whole body image P111 and supply the obtained analysis result to the recommendation server 30 via the image management server 20 together with the whole body image P111. In this case, the recommendation server 30 generates recommendation information R<b>131 based on the analysis result from the photo creation game machine 10 .

Furthermore, the photo creation game machine 10 analyzes the whole body image P111, generates recommendation information R131 based on the obtained analysis results, and sends the generated recommendation information R131 together with the whole body image P111 to the image management server 20. may be supplied to the recommendation server 30 via the In this case, the recommendation server 30 transmits the recommendation information R<b>131 from the photo creation game machine 10 to the portable terminal 40 .

With the above-described flow, recommendation information based on a user's full-body image can be presented to the user on the mobile terminal 40 .

(Presentation of recommended information based on other information)
Recommendation information may be presented to the user based on information other than the captured image generated by the photo creation game machine 10 .

For example, based on the captured image generated by the photo creation game machine 10 and the location information of the store or facility where the photo creation game machine 10 is installed, recommendation information for the user may be generated, or the photo creation game may be generated. Recommendation information for the user may be generated based on the captured image generated by the device 10 and the position information indicating the current position of the mobile terminal 40 .

Further, recommendation information for the user may be generated based on the photographed image generated by the photo creation game machine 10 and the user's member information and profile information (age, gender, etc.) in the image management site or recommendation site.

Furthermore, the high school that the user attends is specified from the uniform worn by the user in the photographed image generated by the photo creation game machine 10, and based on the specified high school, the university or vocational school to be recommended to the user is selected. Representing recommendation information may be generated.

(Use of user data)
The user data stored in the user DB 530 may not only be browsed on the above-described My Page, but may also be reflected in profiles on SNS linked to image acquisition sites and recommendation sites.

(Application on mobile terminal)
Although the recommended information is presented on the web page in the above description, the recommended information may be presented by a dedicated application (dedicated app) installed in the mobile terminal 40 . In this case, the recommendation information may be generated by the recommendation server 30 that has analyzed the image for analysis, as in the above-described embodiment, or may be generated by the recommendation server 30 dedicated to the portable terminal 40 based on the analysis result from the recommendation server 30 . May be generated by the app. In the latter case, the dedicated application of the mobile terminal 40 may further analyze the analysis image.

(Other examples of recommended information)
The recommendation information is not limited to the information for recommending products and services related to makeup, and information for recommending products and services related to fashion. The recommendation information may be, for example, information for recommending the type of plastic surgery to the user, a travel destination, a destination for further education or employment, a destination for moving, and the like.

The series of processes described above can be executed by hardware or by software. When the series of processes described above is executed by software, programs that constitute the software are installed from a network or a recording medium.

For example, as shown in FIG. 3, this recording medium is a magnetic tape recording the program distributed to the administrator of the recommendation server 30 (server 200) separately from the main body of the device. It is composed of a removable medium 119 such as a disk (including a flexible disk), an optical disk (including a CD-ROM and a DVD), a magneto-optical disk, or a semiconductor memory. In addition to these, the recording medium includes the ROM 212 in which the program distributed to the administrator in the state of being preinstalled in the apparatus main body is recorded, the hard disk included in the storage unit 218, and the like.

In this specification, the step of writing a program recorded on a recording medium means not only processing that is performed chronologically in the described order, but also processing that is not necessarily performed chronologically. It also includes processing that is executed individually.

Further, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

Note that the effects described in this specification are merely examples and are not limited, and other effects may be provided.

1 recommendation system, 10 photo creation game machine, 20 image management server, 30 recommendation server, 40 mobile terminal, 131 camera, 132 touch panel monitor, 133 lighting device, 151 printer, 510 control unit, 511 photographing processing unit, 512 image processing unit , 513 print processing unit, 520 control unit, 521 web page provision unit, 522 image analysis unit, 523 recommendation information generation unit, 524 data management unit, 530 user DB

The present technology relates to a server, a recommendation system , and a recommendation method, and more particularly to a server, a recommendation system , and a recommendation method that enable more appropriate recommendations.

A server according to a first aspect of the present technology includes a receiving unit that receives the photographed image from a photograph creation game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a lighting device; a generating unit that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by analyzing a subject area of a captured image; be.
A recommendation system according to a first aspect of the present technology includes a photograph creation game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a lighting device, and an analysis of a subject area of the photographed image. and a server that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by performing the recommendation system.

A server according to a first aspect of the present technology is a photographing game machine that generates a photographed image by photographing a plurality of users as subjects in a photographing space having a camera and a lighting device. Based on the analysis result obtained by analyzing the subject area of the photographed image according to the operation of the receiving unit that receives the image and the mobile terminal owned by each of the users, and a generator that generates recommendation information for recommending at least one of products and services.
A recommendation system according to a first aspect of the present technology includes a photograph creation game machine that generates a photographed image by photographing a plurality of users as subjects in a photographing space having a camera and a lighting device; Recommendation information for recommending at least one of products and services to each of the users based on the analysis result obtained by analyzing the subject area of the captured image according to the operation of the portable terminal. and a server that generates a recommendation system.

In a recommendation method according to a first aspect of the present technology, a photograph creation game machine photographs a plurality of users as subjects in a photographing space having a camera and a lighting device, thereby generating photographed images, which are owned by each of the users. A server recommends at least one of products and services to each of the users based on the analysis result obtained by analyzing the subject area of the photographed image in response to the operation of the portable terminal. This is a recommendation method for generating recommendation information for

A recommendation system according to a second aspect of the present technology includes a photograph creation game machine that generates a photographed image by photographing a plurality of users as subjects in a photographing space having a camera and a light source; A device for recommending at least one of products and services to each of the users, which is generated based on analysis results obtained by analyzing the subject area of the captured image according to an operation on the terminal. and a server for providing recommendation information to the mobile terminal of each of the users.

In the present technology, a photographed image is generated by photographing a plurality of users as subjects in a photographing space having a camera and a lighting device, and the photographed image is reproduced according to the operation of the mobile terminal owned by each of the users. Recommendation information for recommending at least one of products and services to each of the users is generated based on analysis results obtained by analyzing the subject region, or the recommendation information is generated. provided to the mobile terminal of each of the users.

Claims (10)

a photo creation game machine that generates a captured image by capturing a user as a subject in a capturing space having a camera and a lighting device;
A recommendation system comprising: a server that generates recommendation information for recommending at least one of products and services to the user based on analysis results obtained by analyzing the subject area of the captured image. . The photo-creating game machine outputs a sticker sheet on which a processed image obtained by subjecting the subject area of the photographed image to predetermined image processing is printed,
The recommendation system according to claim 1, wherein the server analyzes the subject area by using the captured image that has not undergone the image processing as an image for analysis. The recommendation system according to claim 2, wherein the server analyzes a face image extracted from the analysis image. 4. According to claim 3, the photograph creation game machine generates the photographed image photographed by presenting instruction information indicating a standing position and a face orientation in the photographing space to the user as the image for analysis. Described recommendation system. The photo-creating game machine generates the photographed images photographed by the plurality of users as the analysis image,
The recommendation system according to claim 4, wherein the server analyzes the face image selected by the user from among the plurality of face images extracted from the analysis image. 6. The server generates the recommendation information based on an analysis result obtained by analyzing the face image and an answer selected by the user in response to the question to the user. A recommender system according to any one of . The recommendation system according to claim 6, wherein the recommendation information includes information for recommending at least one of the product and the service related to makeup. The server performs the recommendation for recommending at least one of the product and the service related to fashion based on the analysis result obtained by analyzing the full-body image of the person extracted from the analysis image. 3. The recommender system of claim 2, which generates information. A photo-making game machine
generating a photographed image by photographing a user as a subject in a photographing space having a camera and a lighting device;
the server
A recommendation method for generating recommendation information for recommending at least one of products and services to the user based on analysis results obtained by analyzing a subject area of the captured image. a photography game machine that generates a photographed image by photographing a user as a subject in a photographing space having a camera and a light source;
The user possesses recommendation information for recommending at least one of products and services to the user, which is generated based on analysis results obtained by analyzing the subject area of the captured image. A recommendation system comprising: a server that supplies mobile terminals;

Images