JP2013073420A - Recommendation system, recommendation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To implement a recommendation using attributes of a user by using an acquired user's face image without compelling the user to do pre-registration work.SOLUTION: A portable terminal acquires a user's face image and extracts a feature quantity from the acquired user's face image. Then, the portable terminal transmits the extracted feature quantity to a recommendation server. The recommendation server receives the feature quantity from the portable terminal, stores at least a plurality of feature quantities, performs matching processing of the received feature quantity with the stored feature quantities, recommends an application or the like on the basis of the matching result and transmits the recommendation result to the portable terminal. The portable terminal receives the recommendation result from the recommendation server, and displays the received recommendation result.

Description

  The present invention relates to a recommendation system, a recommendation method, and a program for obtaining a user's face image and performing a recommendation using the user's attribute without causing the user to perform pre-registration work by using the obtained face image.

  2. Description of the Related Art Conventionally, a technique for recommending an application or content that a user would like to use using user attribute information, user history information, and user history information having similar attributes is known.

  By the way, in recent years, tablet terminals are spreading at a remarkable rate. As described above, when tablet terminals are widely used in the home, the form of sharing and using one terminal within a family increases.

  Among them, the service that estimates the attribute of the user who uses the tablet terminal (or recognizes the user), collects the operation log of the user (attribute), and recommends from the estimated result is considered. In order to perform estimation without requiring the user to perform a special operation, use of a face image can be considered.

  Here, as a technique for using a face image, a user's face image is acquired using a camera built in a terminal or the like, a feature amount is extracted from the acquired face image, a person is identified, A technique for limiting possible operations (for example, see Patent Document 1), a feature amount from a face image, a technique for estimating the age of a person (for example, see Patent Document 2), and a feature amount from a face image. Then, a technique for performing attribute estimation, changing a feature amount used for face recognition according to an estimation result, performing face recognition, and specifying a person (for example, see Patent Document 3) is known.

JP 2001-256696 A JP 2009-093490 A JP 2006-02049 A

  However, as in the conventional technology, although there is a certain effect on the recommendation performed only by the result of estimating the user's attribute (gender, age), it is not a recommendation that matches the personal taste of the user, The effect is limited, and there is a problem that logs are collected and analyzed in attribute units.

  In addition, with the above technology that performs face recognition and identifies the user who is using it, it is possible to acquire the user's personal profile (age, gender, favorite thing, etc.) and logs, and recommendations that suit the taste and taste are Although it is possible, for this purpose, there is a problem that the user needs to perform pre-registration work such as registration of a face photo, profile, etc., and the procedure is complicated.

  Therefore, the present invention has been made in view of the above-described problems, and a recommendation system that makes a recommendation using a user attribute without causing the user to perform pre-registration work by using the acquired user face image. It is an object to provide a recommendation method and program.

  The present invention proposes the following items in order to solve the above-described problems. In addition, in order to make an understanding easy, although the code | symbol corresponding to embodiment of this invention is attached | subjected and demonstrated, it is not limited to this.

  (1) The present invention is a recommendation system comprising a portable terminal (for example, equivalent to the portable terminal 100 in FIG. 1) and a recommendation server (for example, equivalent to the recommendation server 200 in FIG. 1), wherein the portable terminal is A face image acquisition unit (for example, corresponding to the image capturing unit 110 in FIG. 2) that acquires a user's face image, and a feature amount extraction unit (for example, the feature in FIG. 2) that extracts a feature amount from the acquired user's face image Equivalent to the quantity extraction unit 140), feature quantity transmission means for transmitting the extracted feature quantity to the recommendation server (for example, equivalent to the transmission / reception unit 170 in FIG. 2), and a recommendation result for receiving the recommendation result from the recommendation server Receiving means (for example, equivalent to the transmission / reception unit 170 in FIG. 2) and display means (for example, equivalent to the information display unit 160 in FIG. 2) for displaying the received recommendation results; A feature amount receiving means (for example, corresponding to the communication unit 250 in FIG. 3), and a storage means for storing at least a plurality of feature amounts. For example, it corresponds to the feature quantity / attribute storage unit 210 in FIG. 3) and a matching processing means for performing a matching process between the received feature quantity and the stored feature quantity (for example, equivalent to the matching processing unit 220 in FIG. 3). ), Recommendation processing means for making a recommendation including at least an application based on the matching result of the matching processing means (for example, equivalent to the recommendation unit 230 in FIG. 3), and a recommendation result of the recommendation processing means to the mobile terminal A recommendation result transmitting means for transmitting (for example, corresponding to the communication unit 250 in FIG. 3). It has proposed a Temu.

  According to this invention, the face image acquisition means of the mobile terminal acquires the user's face image. The feature amount extraction unit extracts a feature amount from the acquired user face image. The feature amount transmitting means transmits the extracted feature amount to the recommendation server. The recommendation result receiving means receives the recommendation result from the recommendation server. The display means displays the received recommendation result. The feature amount receiving means of the recommendation server receives the feature amount from the portable terminal. The storage means stores at least a plurality of feature amounts. The matching processing means performs a matching process between the received feature quantity and the stored feature quantity. The recommendation processing means performs a recommendation including at least an application based on the matching result of the matching processing means. The recommendation result transmission means transmits the recommendation result of the recommendation processing means to the mobile terminal. Therefore, the mobile terminal acquires a face image, extracts a feature amount, and transmits the extracted feature amount to a recommendation server. The recommendation server creates a feature amount database of the face image, and uploads the data Matching (similarity calculation) and making recommendations to the user based on the matching result, it is possible to recommend applications and contents that meet the user's hobbies and preferences without requiring complicated procedures for the user it can.

  (2) The present invention includes a storage unit (for example, equivalent to the image storage unit 120 in FIG. 2) that stores the captured image, and the face image acquisition unit stores the captured image in the recommendation system of (1). The recommendation system characterized by acquiring the said user's face image from among is proposed.

  According to the present invention, the captured image is stored, and the facial image acquisition unit acquires the user's facial image from the stored captured image. Therefore, since the user's face image is acquired from the stored captured image, the recommendation can be executed without bothering the user to acquire his / her face image. There is nothing.

  (3) In the recommendation system according to (2), the face image acquisition unit acquires the face image of the user from the stored captured image using pattern matching. A system is proposed.

  According to this invention, the face image acquisition means acquires a user's face image from the stored captured images using pattern matching. That is, since the face image acquisition means acquires the user's face image by pattern matching such as template matching (face or part constituting the face), the face image of the user can be acquired with high accuracy.

  (4) In the recommendation system according to (2), the face image acquisition unit detects a skin color and acquires the face image of the user from the stored captured image. A system is proposed.

  According to this invention, the face image acquisition means detects the skin color, and acquires the user's face image from the stored captured image. That is, since the face image acquisition means detects the skin color from the captured image and acquires the user's face image, it can acquire the user's face image with high accuracy.

  (5) In the recommendation system of (1), the feature amount extraction unit extracts a facial component from the acquired user's face image, and extracts a feature amount for each extracted component. We propose a recommendation system characterized by this.

  According to the present invention, the feature quantity extraction unit extracts facial components from the acquired user face image, and extracts feature quantities for each of the extracted components. That is, since the facial component is extracted from the acquired user's face image and the feature amount is extracted for each extracted component, the user's attribute information can be acquired with high accuracy.

  (6) The present invention proposes a recommendation system in which the feature quantity extraction unit extracts a Gabor wavelet feature quantity as the feature quantity in the recommendation system of (5).

  According to the present invention, the feature quantity extraction means extracts a Gabor wavelet feature quantity as the feature quantity. Here, the Gabor wavelet feature amount is a feature amount extracted from a so-called Gabor space obtained by applying a Gabor filter to an imaging signal. By using this, a feature amount can be obtained with high accuracy. .

  (7) According to the present invention, in the recommendation system of (1) to (6), the mobile terminal is provided with shaping means for shaping the face image (for example, equivalent to the image shaping unit 180 in FIG. 7), and the shaping The recommendation system characterized by performing the process which extracts the feature-value from a user's face image after the process of a means is proposed.

  According to the present invention, the mobile terminal includes the shaping unit that shapes the face image, and performs the process of extracting the feature amount from the user's face image after the processing of the shaping unit. In other words, the feature amount can be extracted more accurately by performing a normalization process for normalizing the size of the face image or a process using a noise removal filter as the shaping unit.

  (8) In the recommendation system of (1), the present invention provides log acquisition means (for example, in the log acquisition unit 150 of FIG. 1) in which the mobile terminal stores an application operation or usage history in association with the user information. We propose a recommendation system characterized by having

  According to the present invention, the mobile terminal includes the log acquisition means for storing the operation or usage history of the application in association with the user information. Therefore, the recommendation suitable for the user's hobbies and preferences can be executed by using the log.

  (9) According to the present invention, in the recommendation system of (1), the matching processing means calculates similarity by calculating similarity (for example, equivalent to the similarity calculator 221 in FIG. 4), and calculating the similarity. A recommendation system characterized by comprising comparison means (for example, equivalent to the comparison unit 223 in FIG. 4) for comparison with a predetermined threshold value is proposed.

  According to the present invention, the matching processing means includes the similarity calculation means for calculating the similarity and the comparison means for comparing the similarity with the predetermined threshold value. Therefore, the presence or absence of matching can be detected by a simple method by comparing the calculated similarity with a predetermined threshold.

  (10) The present invention proposes a recommendation system in which the similarity calculation means calculates the similarity by pattern matching for the recommendation system of (9).

  According to this invention, the similarity calculation means calculates the similarity by pattern matching. That is, the similarity calculation means calculates the similarity by calculation processing such as likelihood, Mahalanobis distance, and Eugrid distance, and therefore can calculate the similarity with high accuracy.

  (11) In the recommendation system according to (1), the storage unit stores the feature amount, the use history, and the attribute information in association with each other, and the matching processing unit determines that the matching is performed, A recommendation system is proposed in which the usage history and attribute information are output as a matching result to a recommendation processing means.

  According to this invention, the storage means stores the feature quantity, the usage history, and the attribute information in association with each other, and when the matching processing means determines that the matching is performed, the recommendation processing means sends the usage history and the attribute as a matching result. Output information. Therefore, it is possible to execute a recommendation suitable for the user's hobbies and preferences by accurately calculating the similarity.

  (12) In the recommendation system of (1), the present invention provides attribute estimation means (for example, equivalent to the attribute estimation unit 260 in FIG. 3) in which the recommendation server estimates a user attribute based on the feature amount. A recommendation system is proposed in which when the matching processing means does not determine matching, the estimated attribute information is output to the recommendation processing means as a matching result.

  According to this invention, the recommendation server includes the attribute estimation unit that estimates the user attribute based on the feature amount, and when the matching processing unit does not determine the matching feature amount, the recommendation processing unit includes: The estimated attribute information is output as a matching result. Therefore, even when the matching feature amount cannot be extracted, the recommendation according to the user's hobbies and preferences can be executed.

  (13) The present invention is a recommendation method in a recommendation system comprising a portable terminal and a recommendation server, wherein the portable terminal obtains a user's face image (for example, corresponding to step S105 in FIG. 5). ), A second step (for example, corresponding to step S106 in FIG. 5) for extracting a feature amount from the acquired user's face image, and a third step for transmitting the extracted feature amount to the recommendation server ( For example, it corresponds to step S107 in FIG. 5), the fourth step in which the recommendation server receives feature amounts from the mobile terminal (for example, corresponding to step S108 in FIG. 5), and at least a plurality of feature amounts are stored. Then, a fifth step (for example, step S109 in FIG. 5) is performed for matching processing between the received feature amount and the stored feature amount. And a sixth step (for example, corresponding to step S111 in FIG. 5) for performing a recommendation including at least an application based on the matching result in the fifth step, and the recommendation result in the sixth step A seventh step of transmitting to the portable terminal, an eighth step of receiving a recommendation result from the recommendation server (for example, corresponding to step S115 in FIG. 6), and displaying the received recommendation result. A recommendation method characterized by comprising a ninth step (for example, corresponding to step S116 in FIG. 6) is proposed.

  According to this invention, a portable terminal acquires a user's face image, and extracts a feature-value from the acquired user's face image. Next, the extracted feature amount is transmitted to the recommendation server. The recommendation server receives the feature quantity from the mobile terminal, stores at least a plurality of feature quantities, performs a matching process between the received feature quantity and the stored feature quantity, and based on the matching result in the fifth step, The recommendation including at least the application is performed, and the recommendation result in the sixth step is transmitted to the mobile terminal. And a portable terminal receives a recommendation result from a recommendation server, and displays the received recommendation result. Therefore, the mobile terminal acquires a face image, extracts a feature amount, and transmits the extracted feature amount to a recommendation server. The recommendation server creates a feature amount database of the face image, and uploads the data Matching (similarity calculation) and recommending the user based on the matching result, it is possible to recommend applications and contents that meet the user's hobbies and preferences without requiring complicated procedures from the user it can.

  (14) The present invention is a program for causing a computer to execute a recommendation method in a recommendation system composed of a portable terminal and a recommendation server, wherein the portable terminal acquires a user's face image (for example, , Corresponding to step S105 in FIG. 5), a second step (for example, corresponding to step S106 in FIG. 5) for extracting a feature amount from the acquired face image of the user, and the extracted feature amount to the recommendation server A third step (for example, corresponding to step S107 in FIG. 5), and a fourth step (for example, corresponding to step S108 in FIG. 5) in which the recommendation server receives a feature value from the portable terminal; , Storing at least a plurality of feature quantities, and performing a matching process between the received feature quantities and the stored feature quantities. Step (for example, corresponding to step S109 in FIG. 5), and a sixth step (for example, corresponding to step S111 in FIG. 5) for making a recommendation including at least an application based on the matching result in the fifth step, A seventh step of transmitting the recommendation result in the sixth step to the portable terminal, and an eighth step in which the portable terminal receives the recommendation result from the recommendation server (for example, corresponding to step S115 in FIG. 6). And a program for causing a computer to execute a ninth step (for example, corresponding to step S116 in FIG. 6) for displaying the received recommendation result.

  According to this invention, a portable terminal acquires a user's face image, and extracts a feature-value from the acquired user's face image. Next, the extracted feature amount is transmitted to the recommendation server. The recommendation server receives the feature quantity from the mobile terminal, stores at least a plurality of feature quantities, performs a matching process between the received feature quantity and the stored feature quantity, and based on the matching result in the fifth step, The recommendation including at least the application is performed, and the recommendation result in the sixth step is transmitted to the mobile terminal. And a portable terminal receives a recommendation result from a recommendation server, and displays the received recommendation result. Therefore, the mobile terminal acquires a face image, extracts a feature amount, and transmits the extracted feature amount to a recommendation server. The recommendation server creates a feature amount database of the face image, and uploads the data Recommending applications and contents that meet the user's hobbies and preferences without requiring complicated procedures for the user to perform matching (similarity calculation) and recommend the user based on the matching result Can do.

  According to the present invention, by using the acquired user face image, there is an effect that a recommendation using a user attribute can be performed without causing the user to perform pre-registration work.

It is a figure which shows the structure of the recommendation system which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the portable terminal which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the recommendation server which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the matching process part which concerns on the 1st Embodiment of this invention. It is a figure which shows the process of the recommendation system which concerns on the 1st Embodiment of this invention. It is a figure which shows the process of the recommendation system which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the portable terminal which concerns on the 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Note that the constituent elements in the present embodiment can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

<First Embodiment>
A first embodiment of the present invention will be described with reference to FIGS.

<Configuration of recommendation system>
As shown in FIG. 1, the recommendation system according to the present invention includes a mobile terminal 100, a recommendation server 200, and a network 300 for connecting the mobile terminal 100 and the recommendation server 200.

  In the recommendation system according to the present invention, the mobile terminal 100 acquires a face image, extracts a feature amount, and transmits the extracted feature amount to a recommendation server. In the recommendation server, a feature amount database of face images is created, matching with uploaded data (similarity calculation) is performed, and recommendations are made to the user based on the matching result.

<Configuration of mobile terminal>
The configuration of the mobile terminal will be described with reference to FIG.

  As shown in FIG. 2, the portable terminal 100 includes an image capturing unit 110, an image storage unit 120, a face image detection unit 130, a feature amount extraction unit 140, a log acquisition unit 150, an information display unit 160, The transmission / reception part 170 is comprised.

  The image capturing unit 110 is, for example, an in camera built in the mobile terminal 100. The image storage unit 120 stores and saves images captured by the image capturing unit 110. The face image detection unit 130 detects a face image from the images stored and saved in the image storage unit 120. Specifically, the image capturing unit 110 captures an image, temporarily stores it in the image storage unit 120, and repeats capturing and storing until a face is detected in the stored image. Further, the face image detection unit 130 detects a face image by pattern matching such as template matching (a face or a part constituting the face), skin color detection, or the like.

  The feature amount extraction unit 140 extracts facial components such as eyes, nose, mouth, and eyebrows from the face image detected by the face image detection unit 130. For example, pattern matching such as template matching or the like can be used for extraction of the constituent elements of the face. Note that the components to be extracted may be arbitrarily set. And the feature-value in the extracted component is extracted. For example, a Gabor wavelet feature amount can be used to extract the feature amount, but the feature amount is not limited thereto. Then, the extracted feature amount is output to the transmission / reception unit 170.

  The log acquisition unit 150 starts acquisition of a user operation or usage history when the application starts, and ends acquisition of the user operation or usage history when the application ends. The acquired log information is output to the transmission / reception unit 170 together with the user ID.

  The information display unit 160 displays the recommendation result received from the recommendation server 200 on a liquid crystal display device or the like. The transmission / reception unit 170 transmits the feature amount extracted by the feature amount extraction unit 140, the log information output from the log acquisition unit 150, the user ID, and the like to the recommendation server 200, and receives the recommendation result from the recommendation server 200.

<Configuration of recommendation server>
The configuration of the recommendation server will be described with reference to FIG. 3 or FIG.

  As shown in FIG. 3, the recommendation server 200 includes a feature quantity / attribute storage unit 210, a matching processing unit 220, a recommendation unit 230, a log storage unit 240, a communication unit 250, and an attribute estimation unit 260. Has been.

  The feature amount / attribute storage unit 210 stores feature amounts and attributes sent from a matching processing unit 220 and an attribute estimation unit 260 described later, and allocates a user ID. Note that the feature quantity holding method may hold all feature quantities, or may be held as a probability distribution such as average and variance of feature quantities.

  The matching processing unit 220 compares the feature amount received from the mobile terminal 100 with the existing feature amount stored in the feature amount / attribute storage unit 210. Specifically, the matching processing unit 220 includes a similarity calculation unit 221, a threshold storage unit 222, and a comparison unit 223, as shown in FIG. (For example, calculation of likelihood, Mahalanobis distance, Eugrid distance, etc.) and the similarity is calculated. At this time, a threshold value is set in advance, and if the similarity is equal to or higher than this threshold value, it is determined that the same person is used, and if the similarity is equal to or lower than the threshold value, the same person is not determined. If it is determined that they are the same person, the user ID and attribute information stored in the feature quantity / attribute storage unit 210 are sent to the recommendation unit 230. On the other hand, if it is not determined that they are the same person, the feature amount is sent to the attribute estimation unit 260.

  The recommendation unit 230 acquires a past operation or use history from the log storage unit 240 described later based on the input user ID, and inputs it to the recommendation engine together with the attribute information to obtain a recommendation result. At this time, when there is no matching (when there is no user operation / use history). Only attribute information is input to the recommendation engine to obtain a recommendation result. Here, the recommendation engine collects and analyzes use histories and user attributes of a large number of users, and outputs recommended information in response to the input, and any existing method or technology may be used. When the recommendation process is completed, the recommendation result and the user ID are transmitted to the mobile terminal 100 via the communication unit 250.

  The log storage unit 240 stores user operations or usage histories received from the mobile terminal 100 via the communication unit 250. At this time, the received information is stored together with the user ID.

  The communication unit 250 transmits a recommendation result and a user ID to the mobile terminal 100 and receives a user operation or usage history and a user ID from the mobile terminal 100.

  The attribute estimation unit 260 compares the input feature quantity with a class for each attribute (age / gender) learned in advance (for example, calculation of likelihood, Mahalanobis distance, Eugrid distance, etc.), and the most suitable attribute Determine. Then, the estimated attribute and feature amount are sent to the feature amount / attribute storage unit 210, and the user ID is acquired from the feature amount / attribute storage unit 210. Further, the estimated attribute and user ID are sent to the recommendation unit 230.

<Recommendation server processing>
Processing of the recommendation system according to the present embodiment will be described with reference to FIGS. 5 and 6.

  First, the mobile terminal 100 starts an application of the recommendation system according to the present embodiment (Step S101), and displays the application on the liquid crystal display screen of the mobile terminal 100 (Step S102).

  When the application of the recommendation system according to the present embodiment is activated, the control unit of the mobile terminal 100 starts collecting user log information (step S103). Next, when an image is taken by the in-camera of the mobile terminal 100 (step S104), a face image is detected from the taken image (step S105).

  If the face image cannot be detected (“No” in step S105), the process returns to step S104 and waits. If the face image can be detected (“Yes” in step S105), The feature amount is extracted from the detected face image (step S106), and the extracted feature amount and the like are transmitted to the recommendation server 200 (step S107).

  When the recommendation server 200 receives the feature amount extracted from the portable terminal 100 (step S108), the recommendation server 200 performs matching processing between the received feature amount and the existing feature amount (step S109).

  Here, when the received feature value matches the existing feature value (“Yes” in step S109), the received feature value is stored as the existing feature value, and the matched user ID is extracted (step S109). S110), a recommendation process according to the user's history and attributes is executed (step S111).

  On the other hand, if the received feature quantity does not match the existing feature quantity (“No” in step S109), the user attribute is estimated from the received feature quantity (step S112) and saved as the new user feature quantity. Then, the user ID is newly paid out (step S113), and a recommendation process corresponding to the estimated attribute is executed (step S114).

  When the recommendation process in step S111 or step S114 ends, the recommendation server 200 transmits one of the recommendation results and the user ID to the mobile terminal 100, and the mobile terminal 100 transmits the recommendation result and the user ID transmitted by the recommendation server 200. Receive (step S115).

  When the mobile terminal 100 receives the recommendation result from the recommendation server 200, the recommendation result is displayed on the liquid crystal display unit of the mobile terminal 100 (step S116).

  When the application of the recommendation system according to the present embodiment ends (step S117), the mobile terminal 100 ends the user log collection and transmits the collected user log together with the user ID to the recommendation server 200 (step S118). . The recommendation server 200 that has received the user's log and the user ID stores it (step S119).

  As described above, according to the present embodiment, the mobile terminal acquires a face image, extracts a feature amount, transmits the extracted feature amount to a recommendation server, and the recommendation server includes a feature amount of the face image. Create a database, perform matching with uploaded data (similarity calculation), and make recommendations to users based on the matching results. You can recommend applications and content that suit your tastes.

<Second Embodiment>
A second embodiment according to the present invention will be described with reference to FIG. In the present embodiment, the mobile terminal 100 performs a shaping process on the stored image before detecting the user's face image from the stored image.

<Configuration of mobile terminal>
As illustrated in FIG. 7, the mobile terminal 100 according to the present embodiment includes an image capturing unit 110, an image storage unit 120, a face image detection unit 130, a feature amount extraction unit 140, a log acquisition unit 150, information The display unit 160 includes a transmission / reception unit 170 and an image shaping unit 180. In addition, about the component which attaches | subjects the same code | symbol as 1st Embodiment, since it has the same function, the detailed description is abbreviate | omitted.

  Here, the image shaping unit 180 performs image shaping processing such as normalization processing for normalizing the size of the face image and processing for removing noise on the image stored in the image storage unit 120.

<Recommendation server processing>
First, the mobile terminal 100 starts an application of the recommendation system according to the present embodiment, and displays the application on the liquid crystal display screen of the mobile terminal 100.

  When the application of the recommendation system according to the present embodiment is activated, the control unit of the mobile terminal 100 starts collecting user log information. Next, when the in-camera of the mobile terminal 100 captures an image, the captured image is stored, and a normalization process or noise removal is performed to normalize the size of the face image with respect to the stored image. Perform image shaping processing such as processing. When the image shaping process ends, a face image is detected from the image after the image shaping.

  If the face image cannot be detected, the process waits. If the face image can be detected, the feature amount is extracted from the detected face image, and the extracted feature amount and the like are stored in the recommendation server 200. Send.

  When receiving the feature amount extracted from the mobile terminal 100, the recommendation server 200 performs matching processing between the received feature amount and the existing feature amount.

  Here, when the received feature value matches an existing feature value, the received feature value is saved as an existing feature value, and a matching user ID is extracted, and a recommendation according to the user's history and attributes is made. Execute the process.

  On the other hand, if the received feature quantity does not match the existing feature quantity, the user attribute is estimated from the received feature quantity, stored as the new user feature quantity, and the user ID is newly paid out and estimated. Execute the recommendation process according to the attribute.

  When the recommendation process ends, the recommendation server 200 transmits one of the recommendation results to the mobile terminal 100, and the mobile terminal 100 receives the recommendation result transmitted by the recommendation server 200.

  When the mobile terminal 100 receives a recommendation result from the recommendation server 200, the recommendation result is displayed on the liquid crystal display unit of the mobile terminal 100.

  When the application of the recommendation system according to the present embodiment ends, the mobile terminal 100 ends user log collection, and transmits the collected user log together with the user ID to the recommendation server 200. The recommendation server 200 that has received the user's log and user ID stores it.

  As described above, according to the present embodiment, as a shaping unit, a feature amount is more accurately obtained by performing a normalization process that normalizes the size of a face image or the like or a process using a noise removal filter. Can be extracted. In addition, the mobile terminal acquires a face image, extracts a feature amount, and transmits the extracted feature amount to a recommendation server. The recommendation server creates a feature amount database of the face image, and uploads the data Matching (similarity calculation) and making recommendations to the user based on the matching result, it is possible to recommend applications and contents that meet the user's hobbies and preferences without requiring complicated procedures for the user it can.

  Note that the recommendation system of the present invention can be realized by recording the processing of the recommendation system on a computer-readable recording medium, reading the program recorded on the recording medium into the recommendation system, and executing the program. The computer system here includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiments, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100; Portable terminal 110; Image photographing part 120; Image storage part 130; Face image detection part 140; Feature quantity extraction part 150; Log acquisition part 160; Information display part 170; Feature amount / attribute storage unit 220; matching processing unit 221; similarity calculation unit 222; threshold storage unit 223; comparison unit 230; recommendation unit 240; log storage unit 250; communication unit 260; attribute estimation unit 300;

Claims (14)

A recommendation system comprising a mobile terminal and a recommendation server,
The mobile terminal is
Face image acquisition means for acquiring a user's face image;
Feature amount extraction means for extracting a feature amount from the acquired user's face image;
Feature quantity transmitting means for transmitting the extracted feature quantity to the recommendation server;
Recommendation result receiving means for receiving a recommendation result from the recommendation server;
Display means for displaying the received recommendation results;
With
The recommendation server is
Feature quantity receiving means for receiving feature quantities from the portable terminal;
Storage means for storing at least a plurality of feature quantities;
Matching processing means for performing a matching process between the received feature quantity and the stored feature quantity;
A recommendation processing means for performing a recommendation including at least an application based on the matching result of the matching processing means;
Recommendation result transmission means for transmitting a recommendation result of the recommendation processing means to the mobile terminal;
The recommendation system characterized by having. A storage means for storing the captured image;
The recommendation system according to claim 1, wherein the face image acquisition unit acquires the face image of the user from the stored captured image.   The recommendation system according to claim 2, wherein the face image acquisition unit acquires a face image of the user from the stored captured images using pattern matching.   The recommendation system according to claim 2, wherein the face image acquisition unit detects a skin color and acquires the user's face image from the stored captured image.   The recommendation system according to claim 1, wherein the feature amount extraction unit extracts a facial component from the acquired user's face image, and extracts a feature amount for each extracted component.   The recommendation system according to claim 5, wherein the feature quantity extraction unit extracts a Gabor wavelet feature quantity as the feature quantity.   The said portable terminal is provided with the shaping means which shapes the said face image, and performs the process which extracts the feature-value from a user's face image after the process of this shaping means. Recommendation system.   The recommendation system according to claim 1, wherein the portable terminal includes a log acquisition unit that stores an operation or usage history of an application in association with the user information.   The recommendation system according to claim 1, wherein the matching processing unit includes a similarity calculation unit that calculates a similarity, and a comparison unit that compares the similarity with a predetermined threshold.   The recommendation system according to claim 9, wherein the similarity calculation unit calculates the similarity by pattern matching. The storage means stores the feature quantity, usage history, and attribute information in association with each other,
The recommendation system according to claim 1, wherein when the matching processing unit determines that matching is performed, the usage history and attribute information are output to the recommendation processing unit as a matching result.   The recommendation server includes an attribute estimation unit that estimates a user attribute based on the feature amount, and when the matching processing unit cannot determine that the matching is performed, the recommendation processing unit receives the matching result as the matching result. The recommendation system according to claim 1, wherein the estimated attribute information is output. A recommendation method in a recommendation system comprising a mobile terminal and a recommendation server,
The mobile terminal is
A first step of acquiring a user's face image;
A second step of extracting a feature amount from the acquired user's face image;
A third step of transmitting the extracted feature quantity to the recommendation server;
The recommendation server is
A fourth step of receiving feature values from the mobile terminal;
A fifth step of storing at least a plurality of feature quantities and performing a matching process between the received feature quantities and the stored feature quantities;
A sixth step of making a recommendation including at least an application based on the matching result in the fifth step;
A seventh step of transmitting a recommendation result in the sixth step to the mobile terminal;
The mobile terminal is
An eighth step of receiving a recommendation result from the recommendation server;
A ninth step of displaying the received recommendation result;
A recommendation method characterized by comprising: A program for causing a computer to execute a recommendation method in a recommendation system comprising a mobile terminal and a recommendation server,
The mobile terminal is
A first step of acquiring a user's face image;
A second step of extracting a feature amount from the acquired user's face image;
A third step of transmitting the extracted feature quantity to the recommendation server;
The recommendation server is
A fourth step of receiving feature values from the mobile terminal;
A fifth step of storing at least a plurality of feature quantities and performing a matching process between the received feature quantities and the stored feature quantities;
A sixth step of making a recommendation including at least an application based on the matching result in the fifth step;
A seventh step of transmitting a recommendation result in the sixth step to the mobile terminal;
The mobile terminal is
An eighth step of receiving a recommendation result from the recommendation server;
A ninth step of displaying the received recommendation result;
A program that causes a computer to execute.

Images