JP2020038336A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To provide information optimized for each user and provide benefit to each user.SOLUTION: An information processing device comprises: a user information acquisition unit (1); a face information acquisition unit (2); a state detection unit (4) to detect a state of a user; a viewed object determination unit (5); a preference information generation unit (6) to generate preference information of the user; and a program execution unit (30) to execute a program on the basis of the preference information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing device, an information processing method, and an information processing program.

  2. Description of the Related Art In recent years, a method of estimating a customer's taste or the like by comparing a customer's purchasing behavior with a trend of the customer has been used for marketing.

Patent Publication No. 4991440 (registered May 11, 2012)

  However, the prior art as described above accumulates data relating to the purchase history and interests of an unspecified number of customers (users), and is conceived only for use in marketing. It did not consider the benefits to users, such as providing. Further, the trend of each user after the information was provided was not followed, and the provided information could not be evaluated.

  It is an object of one embodiment of the present invention to provide optimized information to each user and to provide benefits to each user.

  Another object of one embodiment of the present invention is to follow a trend of each user after information is provided and evaluate the provided information.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a registered specific user, and a face information acquisition unit that acquires face information of the user. A state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit; and a visual recognition that determines an object visually recognized by the user with reference to a result detected by the state detection unit. An object determination unit, a taste information generation unit that generates taste information about the user's taste with reference to the state of the user by the state detection unit and a determination result by the visible object determination unit, and a program based on the taste information. And a program execution unit for executing.

  Thus, the information processing apparatus can appropriately generate the user's preference information.

  In order to solve the above problem, an information processing device according to an aspect of the present invention includes a visible object determination unit, and the visual object determination unit further includes arrangement information of a page being viewed by the user. By referring to the object, the object visually recognized by the user is determined.

  Thereby, the information processing apparatus 100 can accurately determine the object visually recognized by the user.

  In order to solve the above problem, an information processing device according to an aspect of the present invention includes a visible object determination unit, and the visible object determination unit refers to a result detected by the state detection unit, At least one of the degree of concentration of the user and the emotion of the user is further determined.

  Thus, it is possible to determine how much the user is interested in the object and what kind of emotion the user is viewing the object.

  In order to solve the above problem, the information processing apparatus according to an aspect of the present invention further includes an image acquisition unit that acquires a captured image including a user's field of view from an imaging device worn by the user, The visible object determination unit determines an object visually recognized by the user with reference to the captured image acquired by the image acquisition unit and a result detected by the state detection unit.

  This makes it possible to appropriately determine the object visually recognized by the user.

  According to another embodiment of the present invention, there is provided an information processing apparatus including a program execution unit, wherein the program execution unit is an avatar image corresponding to the user, And generating an avatar image corresponding to at least one of the object visually recognized by the user, the emotion of the user, and the preference information.

  Thereby, the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information based on the preference information to the user.

  This allows the user to enjoy information based on his / her own preferences.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs an evaluation based on the preference information and the proposal information. .

  Thereby, it is possible to evaluate whether the proposal information is appropriate.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit outputs the preference information to an external display device as display data. .

  This makes it possible to visualize the estimated understanding level of the learner.

    In order to solve the above-described problem, the preference information is an estimated understanding level of a user who has answered a certain problem.

  As a result, it is possible to appropriately generate the estimated understanding level of the user.

  In order to solve the above problem, an information processing apparatus according to one embodiment of the present invention includes a program execution unit, and the program execution unit transmits display data including an object in which the estimated understanding level is visualized to an external device. Output to the display device.

  This makes it possible to visualize the estimated understanding level of the learner.

  Further, in order to solve the above problem, an information processing device according to an aspect of the present invention includes a program execution unit, and the program execution unit is configured to display an external display device based on the preference information. Change the display information for.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes the attribute information of the user.

  Thereby, the attribute of the user can be obtained.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit configured to acquire user information, and the user information includes the user in the user information. User identification information is included.

  As a result, another user and the target user can be identified, and the user's taste information can be associated with the user information.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a state detection unit that detects a state of the user from face information acquired by the face information acquisition unit, The state of the user detected by the section is at least one of the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement and chin movement. One.

  Thus, the information processing device can appropriately detect the state of the user.

  According to another embodiment of the present invention, there is provided an information processing apparatus including: a preference information generation unit configured to generate preference information on a preference of the user; and the preference generated by the preference information generation unit. The information is information on the object that the user is interested in and the object that the user is not interested in.

  Thereby, the information of the object with a strong interest and the object with a low interest of each user can be known, and the preference information generation unit can generate the weighted preference information.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a temporary ID assigning unit that assigns a temporary ID to the face information, and determines whether the user has settled the temporary ID. And a collating unit for collating with the settlement information.

  Thus, the information processing apparatus can temporarily distinguish the face information from other face information.

  In order to solve the above problem, an information processing program according to one embodiment of the present invention is an information processing program for causing a computer to function as the information processing device, and causes the computer to function as the preference information generation unit.

  This allows the information processing program to cause the computer to function as the interest information generation unit.

  In order to solve the above-described problem, an information processing method according to an aspect of the present invention is an information processing method executed by an information processing apparatus, and includes a user information acquisition method that acquires user information of a registered specific user. Step, a face information obtaining step of obtaining the user's face information, a state detecting step of detecting the state of the user from the face information obtained in the face information obtaining step, and referring to a result detected in the state detecting step. The preference information relating to the preference of the user is generated with reference to the visible object determination step of determining the object visually recognized by the user, the user information of the user, and the determination result in the visible object determination step. A taste information generating step, and a program execution for executing a program based on the taste information And a step.

  Thereby, it is possible to preferably generate the interest information.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a registered specific user, and a face information acquisition unit that acquires face information of the user. And a preference information acquisition unit that acquires preference information obtained by referring to the user information of the user and the face information of the user.

  Thereby, the information processing apparatus can acquire the user's preference information.

  Further, in order to solve the above problem, an information processing device according to an aspect of the present invention further includes a program execution unit that executes a program based on the preference information.

  Thereby, a program based on the user's preference information can be executed.

  According to another embodiment of the present invention, there is provided an information processing apparatus including a program execution unit, wherein the program execution unit is an avatar image corresponding to the user, And generating an avatar image corresponding to at least one of the object visually recognized by the user, the emotion of the user, and the preference information.

  Thereby, the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information based on the preference information to the user.

  This allows the user to enjoy information based on his / her own preferences.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs an evaluation based on the preference information and the proposal information. .

  Thereby, it is possible to evaluate whether the proposal information is appropriate.

  Further, in order to solve the above problem, an information processing device according to an aspect of the present invention includes a program execution unit, and the program execution unit is configured to display an external display device based on the preference information. Change the display information for.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

  According to another aspect of the present invention, there is provided an information processing apparatus comprising: a state detection unit configured to detect a state of the user from face information acquired by the face information acquisition unit; And a visually recognizable object determining unit that determines an object visually recognized by the user with reference to the detected result.

  Thereby, the information processing apparatus 400 can detect the state of the user and can appropriately determine the object visually recognized by the user.

  In order to solve the above problem, an information processing device according to an aspect of the present invention includes a visible object determination unit that determines an object visually recognized by a user, wherein the visible object determination unit The object visually recognized by the user is determined by further referring to the arrangement information of the page being viewed.

  This makes it possible to accurately determine which object on the page the user is viewing.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a visible object determination unit that determines an object visually recognized by a user, wherein the visible object determination unit performs the state detection. Referring to the result detected by the unit, at least one of the concentration of the user and the emotion of the user is further determined.

  Thus, it is possible to determine how much the user is interested in the object and what kind of emotion the user is viewing the object.

  In order to solve the above problem, the information processing apparatus according to an aspect of the present invention further includes an image acquisition unit that acquires a captured image including a user's field of view from an imaging device worn by the user, The visible object determination unit determines an object visually recognized by the user with reference to the captured image acquired by the image acquisition unit and a result detected by the state detection unit.

  This makes it possible to appropriately determine the object visually recognized by the user.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a state detection unit that detects a state of a user, and the state of the user detected by the state detection unit is the state of the user. Gaze, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and chin movement.

  Thereby, the degree of concentration of the user can be suitably determined.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes the attribute information of the user.

  Thereby, the attribute of the user can be obtained.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit configured to acquire user information, and the user information includes the user in the user information. User identification information is included.

  As a result, another user and the target user can be identified, and the user's taste information can be associated with the user information.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a registered specific user, and a face information acquisition unit that acquires face information of the user. Unit, a user information of the user, and a taste information generation unit that generates taste information regarding the taste of the user with reference to the face information of the user; and a program execution unit that executes a program based on the taste information. , Is provided.

  Thus, the information processing apparatus can appropriately generate the user's preference information.

  According to another embodiment of the present invention, there is provided an information processing apparatus including a program execution unit, wherein the program execution unit is an avatar image corresponding to the user, And generating an avatar image corresponding to at least one of the object visually recognized by the user, the emotion of the user, and the preference information.

  Thereby, the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information based on the preference information to the user.

  This allows the user to enjoy information based on his / her own preferences.

  Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs an evaluation based on the preference information and the proposal information. .

  Thereby, it is possible to evaluate whether the proposal information is appropriate.

  In order to solve the above problem, an information processing device according to one embodiment of the present invention includes a program execution unit, and the program execution unit is configured to display an external display device based on taste information. Change the display information.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

  In order to solve the above problem, an information processing apparatus according to one embodiment of the present invention includes a preference information generation unit that generates preference information, or a preference information acquisition unit that acquires preference information. Is information on the object of strong interest and the object of weak interest of the user.

  Thereby, the information of the object with a strong interest and the object with a low interest of each user can be known, and the preference information generation unit can generate the weighted preference information. In addition, the preference information acquisition unit acquires the preference information, so that the user can acquire the preference information based on his / her own preference.

  According to one embodiment of the present invention, it is possible to provide benefits to each user according to the preferences of each user.

FIG. 2 is a block diagram illustrating components of an information processing apparatus according to Embodiments 1 and 2 of the present invention. It is a figure showing the display of the smart phone provided with the information processor concerning Embodiments 1 and 3 of the present invention. FIG. 9 is a diagram illustrating a display unit of a personal computer including the information processing device according to the first and third embodiments of the present invention. 9 is a table illustrating an example of data output by a visible object determination unit of the information processing device according to Embodiments 1, 3, and 4 of the present invention. 9 is a table illustrating an example of preference information generated by a preference information generation unit of the information processing apparatus according to Embodiments 1, 3, and 4 of the present invention. 5 is a table illustrating an example of an avatar image generated by a program execution unit of the information processing device according to the first embodiment of the present invention. 9 is a table illustrating an example of data output by a program execution unit of the information processing device according to Embodiments 1, 3, and 4 of the present invention. 9 is a table illustrating an example of data output by a program execution unit of the information processing device according to Embodiments 1, 3, and 4 of the present invention. FIG. 4 is a diagram illustrating a display unit of a personal computer including the information processing apparatus according to the first, third, and fourth embodiments of the present invention, wherein FIG. It is. 6 is a flowchart illustrating an example of a processing flow of the information processing apparatus according to the first embodiment of the present invention. 6 is a table illustrating an example of data output by a program execution unit of the information processing device according to the first embodiment of the present invention. 6 is a table illustrating an example of data output by a program execution unit of the information processing device according to the first embodiment of the present invention. It is a figure showing an example of the browser screen which the terminal provided with the information processor concerning Embodiment 2 of the present invention displays. FIG. 14 is a diagram illustrating an example of preference information generated by a preference generation unit of the information processing device according to the second embodiment of the present invention. FIG. 14 is a diagram illustrating an example of preference information generated by a preference generation unit of the information processing device according to the second embodiment of the present invention. FIG. 14 is a diagram illustrating an example of preference information generated by a preference generation unit of the information processing device according to the second embodiment of the present invention. FIG. 13 is a block diagram illustrating components of an information processing device according to a third embodiment of the present invention. FIG. 14 is a diagram illustrating an outline of transmission and reception of preference information between information processing apparatuses according to Embodiments 3 and 4 of the present invention. It is a figure showing an example of the browser screen which the terminal provided with the information processor concerning Embodiment 3 of the present invention displays. FIG. 14 is a block diagram illustrating components of an information processing device according to a fourth embodiment of the present invention.

<First embodiment>
Hereinafter, an embodiment of the present invention will be described in detail. When the configuration of the following specific item (embodiment) is the same as the configuration described in the other items, the description may be omitted. Further, for convenience of explanation, members having the same functions as the members shown in each item are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

[Main Configuration of Information Processing Apparatus 100]
FIG. 1 is a block diagram illustrating components of the information processing apparatus 100. As shown in FIG. 1, the information processing apparatus 100 according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visually recognizable object determination unit 5, an interest information generation unit 6, and a program execution. A section 30 is provided.

  Thereby, the information processing apparatus 100 can suitably generate the user's preference information.

  Here, the state detection unit 4, the visible object determination unit 5, the preference information generation unit 6, and the program execution unit 30 are realized as one configuration of the control unit 9 included in the information processing device 100.

  As an example, the information processing apparatus 100 is mounted on a terminal device (for example, a smartphone, a tablet, a personal computer, a television receiver, or the like) connectable to a local network or a global network. FIG. 2 schematically illustrates a display screen of the smartphone and a line-of-sight position on the display screen when the information processing apparatus 100 is mounted on the smartphone. FIG. 3 schematically shows a display screen of the personal computer when the information processing apparatus 100 is mounted on the personal computer, and a line-of-sight position on the display screen. Other configurations included in FIGS. 2 and 3 will be described later.

  In the present embodiment, a mode in which a user purchases a product using an electronic commerce (EC) service will be described.

(User information acquisition unit)
The user information acquisition unit 1 acquires, for example, user information of a specific user registered in advance in a database of a shopping site or the like, and transmits the acquired user information to the preference information generation unit 6. As an example, the user information includes attribute information indicating the attribute of the user and user identification information for identifying another user and the target user. The attribute information of the user is, for example, the age, gender, type of member (whether or not a paying member) of the user. The user identification information is, for example, a user ID, a user mail address, and the like.

  Thereby, the attribute of the user can be obtained. In addition, since the user information includes the user identification information, another user and the target user can be identified, and the taste information of the user and the user information can be linked.

(Face information acquisition unit)
The face information acquisition unit 2 acquires user face information from an image acquired from an imaging unit such as a camera, for example. The user's face information is information indicating a feature amount of the face. The feature amount of the face refers to, for example, position information indicating the position of each part of the face (eg, eyes, nose, mouth, cheeks, eyebrows, etc.), shape information indicating the shape, size information indicating the size, and the like. . In particular, the eye information is particularly useful because it is possible to evaluate the user's interest and the degree of interest in an object that the user gazes at. The eye information includes, for example, end points of the inner and outer corners of an eye, edges of an iris, a pupil, and the like. Further, the face information acquisition unit 2 may appropriately perform correction processing such as noise reduction and edge enhancement on the image acquired from the imaging unit. The face information acquisition unit 2 transmits the extracted face information to the state detection unit 4.

(State detector)
The state detection unit 4 detects the state of the user based on the face information acquired by the face information acquisition unit 2. After detecting the state of the user, the state detection unit 4 transmits the detection result to the visible object determination unit 5. The state of the user detected by the state detection unit 4 is, for example, the state of each part of the user's face. The user's line of sight, pupil state, number of blinks, movement of eyebrows, movement of cheeks, movement of eyelids At least one of movement, lip movement, and chin movement.

  As an example, the state detection unit 4 outputs position information and a shape indicating the position of each part of the face (for example, eyes, nose, mouth, cheeks, eyebrows, and the like) which are the feature amounts of the face acquired by the face information acquisition unit 2. Referring to the shape information indicating the size and the size information indicating the size, for example, the user's gaze, pupil state, number of blinks, eyebrow movement, cheek movement, eyelid movement, lip movement At least one of movement and jaw movement may be detected.

  Thus, the information processing device can appropriately detect the state of the user.

  For example, the method of detecting the line of sight is not particularly limited, but a point light source (not shown) is provided in the information processing apparatus 100, and a corneal reflection image of light from the point light source is photographed by the imaging unit for a predetermined time, so that the user A method of detecting the moving destination of the line of sight of the camera is described. The type of the point light source is not particularly limited, and includes visible light and infrared light. For example, by using an infrared LED, it is possible to detect the line of sight without giving the user any discomfort. In the detection of the line of sight, if the line of sight does not move for a predetermined time or more, it can be said that the same place is being watched.

  The method of detecting the state of the pupil is not particularly limited, and includes, for example, a method of detecting a circular pupil from an eye image using Hough transform. In general, humans tend to open their pupils when they are concentrated. Therefore, by detecting the size of the pupil, the degree of concentration of the user can be evaluated. For example, when the size of the pupil is detected for a predetermined time and the pupil is large within the predetermined time, it can be said that there is a high possibility that the user is watching the target. A threshold value may be set for the pupil size, and evaluation may be made as “open” when the pupil size is equal to or larger than the threshold value, and as “closed” when the pupil size is less than the threshold value.

  Further, the method of detecting the number of blinks is not particularly limited. For example, infrared light is applied to the user's eyes, and the difference between the amount of reflected infrared light between the time when the eye is opened and the time when the eye is closed is detected. Method and the like. In general, when a person is concentrated, the person tends to blink at a stable interval at a low frequency. Therefore, the degree of concentration of the user can be evaluated by detecting the number of blinks. For example, if the number of blinks is detected for a predetermined time and the blinks are performed at a stable interval within the predetermined time, it can be said that there is a high possibility that the user is watching the target.

  If the state detection unit 4 detects at least one of the user's line of sight, the state of the pupil and the number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement and jaw movement Good, but it is preferable to combine them. By combining the detection methods in this manner, the state detection unit 4 can appropriately evaluate the degree of concentration of the user when viewing a certain target object.

  Other than the eye condition, for example, the movement of the eyebrows such as whether to lift the inside or the outside of the eyebrows, the movement of the eyelids such as raising the upper eyelid or tightening the eyelids, the movement of the nose such as wrinkling the nose The state of each part of the face such as lip movement such as raising the upper lip and pursing the lips, cheek movement such as raising the cheek, and jaw movement such as lowering the chin. As the state of the user, the states of a plurality of parts of the face may be combined.

  When detecting the user's facial expression, the state detecting unit 4 can also calculate information on the user's facial expression by machine learning. Although the specific configuration of the learning process for acquiring information on facial expressions is not limited to the present embodiment, for example, any of the following machine learning methods or a combination thereof can be used.

・ Support Vector Machine (SVM)
・ Clustering
・ Inductive Logic Programming (ILP)
・ Genetic Algorithm (GP)
・ Bayesian Network (BN)
・ Neural Network (NN)
When a neural network is used, it is preferable to process data beforehand for input to the neural network. For such processing, in addition to one-dimensional data arrangement or multi-dimensional data arrangement, for example, a technique such as data argumentation (Deta Argumentation) can be used.

  When a neural network is used, a convolutional neural network (CNN: Convolutional Neural Network) including a convolution process may be used. More specifically, a convolution layer for performing a convolution operation is provided as one or a plurality of layers (layers) included in the neural network, and a filter operation (product-sum operation) is performed on input data input to the layer. It may be configured. When performing the filter operation, processing such as padding may be used in combination, or an appropriately set stride width may be employed.

  Further, as the neural network, a multi-layer or super-multilayer neural network having several tens to thousands of layers may be used.

(Visual object determination unit)
The visible object determination unit 5 determines the object visually recognized by the user by further referring to the layout information of the page being viewed by the user.

  Thereby, the information processing apparatus 100 can accurately determine the object visually recognized by the user.

  The specific processing of the visually recognizable object determination unit 5 will be described with reference to FIGS. FIG. 2 illustrates a smartphone including the information processing apparatus 100, and FIG. 3 illustrates a personal computer including the information processing apparatus 100.

  2, the terminal device (smartphone) 10 including the information processing device 100 according to the present embodiment further includes an imaging unit 14 and a display unit 12. For example, as shown in FIG. 2, it is assumed that the display area of the browser is the same as the size of the display unit of the smartphone, and the images of the products are arranged side by side. For example, the display order of the products, the manufacturer of each product, and the product name can be extracted from the source code of the web page displayed on the display unit 12. Since each product image area is controlled by the browser, the visible object determination unit 5 can acquire the coordinates of the image area of each product by referring to the source code and the browser. The visible object determination unit 5 collates the acquired coordinates of the image area of each product with the line-of-sight information obtained from the state detection unit 4 so that the position coordinates of the user's line of sight are within the coordinates of the image area of any product. It is determined whether there is. For example, as illustrated in FIG. 2, when it is determined that the position coordinates of the user's line of sight are within the coordinates of the image area 18 of the product a, the visual recognition object determination unit 5 sets the position coordinates of the user's line of sight to: The counting of the time of staying within the coordinates of the image area of the product a is started. In this way, by comparing the position coordinates of the user's line of sight with the coordinates of the product image area, it is possible to suitably determine which product the user is viewing. In addition to the line of sight information, as described in (State Detection Unit), the state of the pupil, the number of blinks, the movement of the eyebrows, the movement of the eyelids, the movement of the cheeks, the movement of the nose, the movement of the lips, and the movement of the chin By referring to the motion detection result, it is possible to more appropriately determine which product the user is focusing on and viewing.

  3, the terminal device (personal computer) 10 including the information processing device 100 according to the present embodiment further includes an imaging unit 14 and a display unit 12, as in FIG. In the case of FIG. 3, since the size of the browser screen 16 is smaller than that of the display unit 12, the visually recognizable object determination unit 5 also needs to acquire information on the position of the browser screen 16 on the display unit 12. Since the position and area of the browser are controlled by the OS (Operating System) of the information processing apparatus 100, the visual recognition object determination unit 5 refers to the OS in addition to the source code and the browser, and refers to the coordinates of the image area of each product. Can be obtained. For example, as illustrated in FIG. 3, when it is determined that the position coordinates of the user's line of sight are within the coordinates of the image area 18 of the product b, the visual recognition object determination unit 5 determines that the position coordinates of the user's line of sight are: The counting of the time of staying within the coordinates of the image area of the product b is started.

  Further, the visible object determination unit 5 may further determine at least one of the degree of concentration of the user and the emotion of the user with reference to the result acquired from the state detection unit 4.

  Thus, it is possible to determine how much the user is interested in the object and what kind of emotion the user is viewing the object.

  The visible object determination unit 5 determines the user's emotion from the state of each part of the user's face acquired from the state detection unit 4. To determine the user's emotion, for example, a Paul Ekman emotion classification model may be used. The emotion classification model includes, for example, “happiness”, “surprise”, “anger”, “disgust”, “fear”, and “sadness”. The emotion classification model may include “normal”. Further, the emotion classification model includes “fun”, “contempt”, “satisfaction”, “confused”, “excited”, “guilt”, “pride based on achievement”, “security”, “satisfaction”, It may include emotions such as “shame” and a combination of these emotions (eg, anger + disgust, happiness + satisfaction, etc.).

  In addition, the visual recognition object determination unit 5 may determine whether the user concentrates on which product and visually recognizes the user and the determination of the user's emotion using the machine learning method described in (state detection unit). Good.

  The visible object determination unit 5 transmits the result of the determination to the preference information generation unit 6.

(Image acquisition unit)
The information processing device 100 may further include an image acquisition unit 15 that acquires a captured image including the user's field of view from an imaging device worn by the user. Here, the imaging device worn by the user is not particularly limited as long as the user's state can be detected. Examples of the imaging device include a camera mounted on a glasses-type wearable device, a camera mounted on a head-mounted device, and the like. These devices may be of a transmission type or a non-transmission type. In the case of the non-transmissive type, as an example, a configuration in which a camera (imaging device) and a display (display unit) ahead of which the user is viewing may be provided in the same device.

  The captured image including the user's field of view is, for example, an image obtained by capturing the view of the destination viewed by the user, the display (display unit), and the like.

  The visible object determination unit 5 may determine the object visually recognized by the user with reference to the captured image acquired by the image acquisition unit 15 and the result detected by the state detection unit 4. Specifically, the visual recognition object determination unit 5 collates the coordinates of the captured image area with the user's line of sight information detected by the state detection unit 4, and determines the position coordinates of the user's line of sight to which coordinates in the captured image. Determine whether they correspond. The visible object determination unit 5 may be configured to recognize an object included in the captured image acquired by the image acquisition unit 15 using, for example, machine learning.

  This makes it possible to appropriately determine the object visually recognized by the user.

(Preference information generation unit)
The preference information generation unit 6 refers to the user information acquired from the user information acquisition unit 1 and the determination result acquired from the visible object determination unit 5 to generate preference information on the user's preference. Here, the preference information is information indicating which objects are more interested in the user and which objects are weaker.

  As a result, it is possible to know the information of the object that each user has a strong interest and the object that the user is weakly interested, and the taste information generating unit 6 can generate the weighted taste information.

  For example, the preference information generation unit 6 may determine the information of the product visually recognized by the user, the result of the user's concentration when viewing the product, and the result of the user's emotion, which are determined by the visually recognized object determination unit 5. refer.

  FIG. 4 is data input to the interest information generation unit 6 and showing the result detected by the state detection unit 4. FIG. 4 is an example of data on a user whose ID is A (hereinafter, referred to as user A). The table in FIG. 4 shows, for each product, the time when the user A visually recognized the product (“line of sight”), the number of blinks per minute (“blink”), and the opening and closing of the pupil (“pupil state”). ing. From the table in FIG. 4, it is determined that the user A has visually recognized the product a for a long time and is in a highly concentrated state based on the number of blinks and the state of the pupil during that time. In addition, from the table of FIG. 4, it is determined that the user A has a short viewing time of the product c with respect to the product c and is not in a very high concentration state.

  The table of FIG. 5 is an example of the preference information generated for each user's attribute, a product with strong interest, and a product with low interest. Here, the attribute of each user is, for example, gender. The preference information generation unit 6 generates preference information as shown in FIG. 5 with reference to the data in the table of FIG. The table in FIG. 5 indicates that the user A is a male, and that the user A is actually strongly interested in the product a and is not so interested in the product b. Such preference information may be information indicating the preference of each user, or may be information indicating the preference of a plurality of users as shown in FIG.

  Further, the preference information generation unit 6 may generate the preference information by using a machine learning method described in (State Detection Unit).

  Further, the preference information generation unit 6 may be configured to generate the user's preference information by referring to other information such as click information together with or instead of the various information described above. Here, the click information may include information such as the position clicked or touched by the user, the number of clicks or touches, and the time interval between double clicks. Further, a configuration may be adopted in which an intensity detection unit that detects the strength of the click or touch is provided, and the taste information generation unit 6 generates the taste information of the user with reference to the strength of the click or touch.

  When the display unit is configured as a touch panel, the strength of a user's touch can be detected by employing a so-called resistive film system.

  The preference information generation unit 6 transmits the generated preference information to the program execution unit 30.

(Program execution unit 1-1)
The program execution unit 30 generates an avatar image corresponding to the user, and detects the state of the user detected by the state detection unit 4, the object visually recognized by the user determined by the visually recognized object determination unit 5, the emotion of the user, and the taste information. An avatar image corresponding to at least one of the preference information acquired from the generation unit 6 is generated.

  The avatar image is information that symbolically represents the user, and is mainly an image that reminds the user. When the user has selected a product at the shopping site, for example, the avatar image is generated on an image formed by imitating a real store.

  Thereby, the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  The user's state is the state of each part of the user's face described in (state detection unit). The object visually recognized by the user and the emotion of the user are determined by the method described in (Visually Recognized Object Determination Unit). Further, the preference information is information indicating which object the user's interest is strong and which object is weak, as described in (Preference Information Generation Unit).

  FIG. 6 is an example of an avatar image generated by the program execution unit 30. FIG. 6A is an example of a conventional avatar image, FIG. 6B is an example of an avatar image according to the user's facial expression, and FIG. 6C is an example of an avatar image according to the user's facial expression and eyes. . As shown in FIG. 6A, the conventional avatar image does not reflect the expression of the user. As shown in FIG. 6B, the program execution unit 30 according to the present embodiment generates an avatar image according to the expression of the user. For example, when the user is viewing a product and has a facial expression with a raised corner, an avatar image of the facial expression with a raised corner is generated. In addition, the program execution unit 30 may display a gauge that visualizes the degree of preference (preference) of the user for a certain product by referring to the preference information. As shown in FIG. 6C, an avatar image may be generated according to the information of the user's line of sight. The line of sight information is, for example, the direction of the line of sight, the state of the pupil, the number of blinks, and the like. For example, when the user is visually recognizing the image of the product a on the shopping site, on an image in which the product is displayed as in an actual store, the avatar looks at the direction of the displayed product a, The sight line of the avatar image may be set. Further, the program execution unit 30 may generate an image in which the object which the user is visually recognizing is held in the avatar's hand, for example, and the product which the avatar has in the hand is visually recognizable.

  The avatar image may be displayed on the display unit 12 of the terminal device of the user so that the user can view the avatar image, or the avatar image of the user may be viewed by another user. The information may be transmitted to another terminal device, a server of a shopping site, or the like.

  Thereby, the user can objectively view the situation of shopping at the shopping site as if he / she is shopping at an actual store. Further, other users (such as store staff) can regard the user shopping at the shopping site as one of the customers shopping at the actual store via the avatar image, and provide a better service to the user. Awareness for providing can be improved.

(Program execution unit 1-2)
The program execution unit 30 may output the preference information generated by the preference information generation unit 6 to an external display device as display data. Although the type of the display data is not particularly limited, for example, a numerical value, a character, a table, a graph, an animation, and the like can be given. An example of the table is, for example, a table as shown in FIG. The preference information to be output may be the preference information of one specific user or the preference information of a plurality of users.

  This makes it possible to visualize user preference information.

(Program execution unit 1-3)
The program execution unit 30 proposes proposal information to the user based on the preference information acquired from the preference information generation unit 6. The proposal information is, for example, an electronic announcement of a product having a strong interest in the user's preference information, information on a price drop, an electronic coupon, and the like. The program execution unit 30 may transmit the generated proposal information to the user using communication means such as an email, a social network service (SNS), and an instant message (IM). The user may be notified as a banner advertisement.

  FIG. 7 is a table illustrating an example of the proposal information generated by the program execution unit 30 based on the preference information acquired from the preference information generation unit 6. As an example, the taste information generation unit 6 estimates that the users A, C, and D have a strong interest in the product a, and the taste information that the users E and F have a strong interest in the product b. Generate. In this case, the program execution unit 30 transmits the advertisement of the product a (proposal information) to the users A, C, and D based on the preference information, and transmits the advertisement of the product b to the users E and F.

  In this way, the program execution unit 30 proposes the proposal information to the user, so that the user can enjoy the information based on his / her own taste.

  The program execution unit 30 may determine whether or not the user has visually recognized the proposal information. For example, the program execution unit 30 may determine whether or not the user has visually recognized whether or not the user has accessed the address of the shopping site described in the advertisement. Further, for example, the program execution unit 30 may determine by referring to information on whether or not the user has logged in to the shopping site within a predetermined period after the advertisement is distributed. Information as to whether or not the user has logged in to the shopping site can be acquired from the user information acquisition unit 1, for example. When logging in to the shopping site, the program execution unit 30 refers to the information on which product image the user has viewed to determine whether the user is viewing the same product image as the product described in the advertisement. It may be determined. Information on which product image the user has visually recognized may be determined by, for example, the visible object determination unit 5 using the method described in (visual object determination unit).

  Further, the program execution unit 30 may perform the evaluation based on the preference information generated by the preference information generation unit 6 and the proposal information proposed by the program execution unit 30. The evaluation method will be described below. Here, the evaluation is, for example, an evaluation of whether the proposal information proposed to the user is appropriate.

  FIG. 8 is a table showing an example of the evaluation performed by the program execution unit 30. The table of FIG. 8 shows that, when the program execution unit 30 transmits an advertisement of the product a to the users A, C, and D and transmits an advertisement of the product b to the users E and F, each user displays the product image of the advertisement. This is the result of evaluation based on the result of the determination as to whether or not visual recognition has been performed. In the present embodiment, the evaluation performed by the program execution unit 30 may be, for example, a ratio of the number of users who have viewed the advertisement to the number of all users who have transmitted the advertisement as the evaluation value. Here, the evaluation value is represented, for example, as “attention rate”. For example, among the users A, C, and D who transmitted the advertisement of the product a, since it was the users A and C that the program execution unit 30 determined that the advertisement was visually recognized, the program execution unit 30 Is 66%. In addition, among the users E and F who transmitted the advertisement of the product b, only the user F determined that the program execution unit 30 has visually recognized the advertisement, and thus the program execution unit 30 sets the attention rate to 50% Is evaluated.

  As described above, the evaluation by the program execution unit 30 enables the evaluation as to whether the proposal information is appropriate.

(Program execution unit 1-4)
The program execution unit 30 changes display information to be displayed on an external display device based on the preference information acquired from the preference information generation unit 6. Here, the external display device is, for example, the display unit 12 of the terminal device. The display information includes, for example, image information indicating an image displayed on the display unit 12, layout information indicating coordinates of the image, and the like. The display information can be obtained from a source code, a browser, and an OS. The change of the information is not particularly limited, but is, for example, a change of the layout of the product image displayed on the display unit 12. The program execution unit 30 transmits the changed display information to the display unit 12.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

  A specific example of the process performed by the program execution unit 30 will be described. FIG. 9 shows the display unit 12 of the personal computer. FIG. 9A shows the display unit 12 before the display is changed, and the display unit 12 displays images of products a to f. FIG. 9B shows the display unit 12 after the display has been changed.

  Here, for example, it is assumed that the preference information generation unit 6 generates the preference information indicating that the commodities a to f that are of great interest to the user are d, e, and f. The program execution unit 30 determines the positions of the images of the products a to c and the images of the products df based on the currently displayed display information in FIG. 9A and the preference information generated by the preference information generation unit 6. The display information in which the layout is changed by replacing the position with the position is generated. The display unit 12 makes a display as shown in FIG. 9B based on the changed display information acquired from the program execution unit 30. That is, the program execution unit 30 arranges and displays the images of the products d, e, and f in the most prominent places on the displayed screen. Note that the change of the layout is not limited to this mode, and may include, for example, switching of product images up and down, and switching of left and right. Further, based on the preference information of the preference information generation unit 6, the images may be rearranged one by one in descending order of the interest of the user.

  The timing at which the display information on the display unit 12 is changed is not particularly limited, but may be the timing at which the web page is updated, or may be changed at predetermined time intervals. Further, the change may be made when the user logs out once from the shopping site or the like and then logs in to the same site.

In the above description, the change of the layout information included in the display information has been described as an example of the process of changing the display information by the program execution unit 30. However, the present embodiment is not limited to this. As another example of the change of the display information, there is a configuration in which an image of a product that the user is interested in is highlighted. Specifically, as an example of highlighting,
-Frame the image of the product-Change the background color around the image of the product-Display the image of the product larger than other products-Display the image of the product as a pop-up window No.

(Control unit)
The control unit 9 controls each unit of the information processing apparatus 100 in an integrated manner. The control unit 9 includes a state detection unit 4, a visible object determination unit 5, an interest information generation unit 6, and a program execution unit 30. Each function of the control unit 9 and all the functions included in the information processing apparatus 100 may be realized by, for example, the CPU executing a program stored in a storage unit (not shown) or the like.

[Processing example 1 of information processing apparatus]
Next, processing of the information processing apparatus 100 will be described based on the flowchart of FIG.

  First, use of the information processing apparatus is started (step S100), and processing is started. The process proceeds to step S (hereinafter, “step” is omitted) 102.

  In S102, the user information acquisition unit 1 acquires user information (user information acquisition step). The details of the processing are as described in (User Information Acquisition Unit). The user information acquisition unit 1 transmits the user information to the preference information generation unit 6, and proceeds to S104.

  In S104, the face information acquisition unit 2 acquires the user's face image from the imaging unit, and extracts each part of the face from the face image (face information acquisition step). The details of the processing are as described in (Face Information Acquisition Unit). The face information acquisition unit 2 transmits the extracted result to the state detection unit 4, and proceeds to S106.

  In S106, the state detection unit 4 detects the user's line of sight, blinks, the state of the pupil, and the state of each part of the face (state detection step). The details of the processing are as described in (Status Detector). The detection result is transmitted to the visible object determination unit 5, and the process proceeds to S108.

  In S108, the visible object determination unit 5 determines the product image (visual object) visually recognized by the user, the degree of concentration when the user is visually recognizing, and the emotion when the user is visually recognizing (visual object determination step). The details of the processing are as described in (Visual recognition object determination unit). The visible object determination unit 5 transmits the result of the determination to the preference information generation unit 6, and proceeds to S110.

  In S110, the preference information generation unit 6 generates preference information with reference to the user information acquired from the user information acquisition unit 1 and the determination result acquired from the visual recognition object determination unit 5 (a preference information generation step). , S112. The details of the processing are as described in (Preference Information Generation Unit).

  In S112, the control unit 9 records the preference information generated by the preference information generation unit 6 in a database or the like, and proceeds to S114.

  In S114, the program execution unit 30 performs at least one of generation of an avatar image, proposal of proposal information, evaluation, and change of display information (program execution step). The details of the processing performed by the program execution unit 30 are as described in (Program execution unit 1-1) to (Program execution unit 1-3). Proceeding to S116, the preference estimation process ends.

  Thereby, it is possible to preferably generate the interest information.

(Modification 1-1)
The information processing device 100 may further include a payment information acquisition unit (not shown). The payment information obtaining unit obtains, for example, payment information indicating whether or not the user has purchased a product. The settlement information can be acquired from, for example, credit card information registered in advance at a shopping site, a settlement server of a shopping site, a settlement server of a settlement agency, and the like. The payment information acquisition unit can acquire information on which product the user has purchased in the shopping site. The payment information is not limited to the credit card information, but can be obtained from a medium associated with the user's personal information. For example, there are debit card information, prepaid card information, and the like. The payment information acquisition unit transmits the acquired payment information to the preference information generation unit 6.

  The settlement information is information on whether or not a product has been purchased in shopping or the like, but is not limited to this, and is more generally, decision-making information that reflects the intention of the user.

  With such a configuration, the information processing apparatus 100 can generate more accurate preference information.

  In addition, since the information processing apparatus 100 includes the payment information acquisition unit, the program execution unit 30 determines whether the user has purchased a product based on the proposal information, and performs an evaluation based on the determination result. be able to.

  FIG. 11 shows a case where the program execution unit 30 transmits an advertisement (proposal information) of the product a to the users A, C, and D and transmits an advertisement of the product b to the users E and F. It is a result of evaluating whether or not a purchase has been made. The contract rate in the table of FIG. 11 is a ratio of the number of users who actually purchased the advertisement product to the number of users to whom the program execution unit 30 transmitted the advertisement. Since all the users A, C and D who have transmitted the advertisement of the product a have purchased the product a, the program execution unit 30 evaluates that the closing rate is 100%. For users E and F that have transmitted the advertisement of the product b, only the user E has purchased the product b, so the program execution unit 30 evaluates that the closing rate is 50%. At this time, it is not always necessary to determine whether the user who has transmitted the advertisement has actually viewed the advertisement. In particular, in an advertisement such as an e-mail that can be confirmed even in an offline state, the determination need not be performed.

  In addition, the program execution unit 30 may perform the evaluation with reference to the advertisement viewing information indicating whether the user who has received the advertisement has actually viewed the advertisement.

  FIG. 12 shows that when the program execution unit 30 transmits the advertisement of the product a to the users A, C, and D, and transmits the advertisement of the product b to the users E and F, each user reaches the purchase of the advertisement. It is a figure showing the result of having evaluated whether or not. The contract rate in the table of FIG. 12 is a ratio of the number of users who have visually recognized the advertisement and have purchased the product to the number of users who have transmitted the advertisement by the program execution unit 30. As shown in the table of FIG. 12, all of the users A, C and D who transmitted the advertisement of the product a purchased the product a, but only the users A and C saw the advertisement and made a purchase. Therefore, the program execution unit 30 evaluates that the closing rate is 66%. Further, as for the users E and F who transmitted the advertisement of the product b, since there was no user who has purchased the product after seeing the advertisement, the program execution unit 30 evaluates that the closing rate is 0%.

  The viewing information of the advertisement may determine whether or not the advertisement has been viewed by using the method described in the (viewing object determination unit).

(Modification 1-2)
The information processing apparatus 100 according to this modification includes the same units as the information processing apparatus 100 according to the first embodiment, and the processing performed by each unit is the same except for the visible object determination unit 5. The visible object determination unit 5 will be described later. Further, the information processing apparatus 100 according to the present embodiment includes a payment information acquisition unit (not shown) that acquires user's payment information, a temporary ID assignment unit (not shown) that assigns a temporary ID, a temporary ID, The information processing apparatus 100 according to the first embodiment is different from the information processing apparatus 100 according to the first embodiment in further including a matching unit (not shown) for matching the payment information.

  By including the payment information acquisition unit, the information processing apparatus 100 can generate more accurate preference information. With the provision of the temporary ID assignment unit, the information processing apparatus 100 can temporarily distinguish face information from other face information. In addition, the provision of the collation unit enables collation between the temporary ID and the payment information of the user.

  In the first embodiment, the mode in which the user purchases a product using the EC service has been described. In the present modification, the mode in which the user purchases a product at an actual store will be described.

  In the first embodiment, the information processing apparatus 100 first acquires personal information of a user. This corresponds to, for example, a user inputting an ID and a password at a shopping site to log in to the shopping site. However, in a real shop, the user rarely logs in beforehand.

  In the present modification, for example, the information processing apparatus acquires the user information after acquiring the settlement information from the settlement information acquiring unit or simultaneously with the settlement information. As an example, an imaging unit is installed on a display shelf or the like of a product in a store so that the user's line of sight, blinking, pupil state, and the like when the user selects a product can be detected. Until the user information is obtained, for example, a temporary ID assigning unit assigns a temporary ID to the user, and stores information on a product visually recognized by the user as temporary ID information in a cache memory or the like. You may. When a user makes a payment at a cash register using a medium to which user information is linked, such as a credit card, payment information and user information are obtained for the first time. For example, an imaging unit is also installed at a cash register, and an image obtained from the imaging unit on the display shelf is synchronized with an image obtained from the imaging unit on the cash register, and which temporary ID user has settled the payment. The determination may be made, and the collating unit may collate the temporary ID information with the payment information of the user or the temporary ID information with the user information.

  In addition, when the user of the temporary ID has settled in a method that cannot be compared with the user information, such as when the user has settled with cash, the temporary ID information may be discarded.

<Embodiment 2>
The information processing device according to the present embodiment is a learning support device. As a form of the learning support device, for example, a device for performing E-learning can be cited. In the present embodiment, the information processing device generates an estimated understanding level of a learner who answers a plurality of choice questions.

  FIG. 13 is an example of a question sentence and an answer option displayed by the information processing apparatus according to the present embodiment. As shown in FIG. 13, the learner reads the displayed question sentence, reads the answer options, and then selects the option that is considered to be the correct answer. In such a multiple-choice question, even if the learner finally selects the correct answer, the learner does not always have to correctly understand the problem and correct the answer. In some cases, the correct answer is accidentally obtained as a result of being lost in a plurality of options, and in other cases, the correct answer is accidentally obtained as a result of selecting an option without reading the question sentence. The information processing apparatus according to the present embodiment outputs the estimated understanding level of the learner in consideration of the process until the learner reaches the answer. As described above, by outputting the estimated understanding level, the learner himself or a third party who instructs the learner can know the understanding level of the learner accurately.

  As in the first embodiment, the information processing apparatus according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, an interest information generation unit 6, And a program execution unit 30.

  The user information acquisition unit 1 acquires information of the user, with a previously registered learner as a user.

  The face information acquisition unit 2 acquires user face information. The acquisition of the user's face information performed by the face information acquisition unit 2 may use the same processing as the face information acquisition unit 2 of the first embodiment.

  The state detection unit 4 detects the state of the user from the user's face information acquired by the face information acquisition unit 2. As the detection method, the same determination method as that of the state detection unit 4 of the first embodiment may be used.

  The visible object determination unit 5 refers to the result detected by the state detection unit 4 and determines the object visually recognized by the user. The visible object determination unit 5 may determine the object visually recognized by the user by further referring to the layout information of the page being viewed by the user.

  In addition, the visible object determination unit 5 may further determine at least one of the degree of concentration of the user and the emotion of the user with reference to the result detected by the state detection unit 4. The determination method performed by the visible object determination unit 5 may use the same method as the determination method of the visible object determination unit 5 of the first embodiment. In the present embodiment, the object visually recognized by the user is a question sentence and an option of an answer visually recognized by the user. In the present embodiment, the emotion of the user is, for example, the emotion of the learner when solving a problem such as being confident or in trouble.

  In the present embodiment, the preference information generated by the preference information generation unit 6 is an estimated understanding level of a user who has answered a certain question. Here, the estimated degree of understanding is a degree for estimating the degree of understanding of the learner (user) in consideration of only the process up to the answer. The interest information generating unit 6 refers to the user information obtained from the user information obtaining unit 1 and the determination result obtained from the visual recognition object determining unit 5 to generate an estimated understanding degree of the learner. As an example, for the object that the visual recognition object determination unit 5 has determined that the learner has turned his or her eyes many times, that is, for the answer option that has been read back many times, the taste information generation unit 6 determines that the learner has a correct answer. Is determined to be a lost option as a candidate, and an estimated understanding level is generated. In the present embodiment, the preference information generation unit 6 generates an estimated understanding level of the learner from the user information and the determination result without considering the result of which option the user has actually selected.

  As a result, it is possible to appropriately generate the estimated understanding level of the learner.

  FIG. 14 is an example of information generated by the preference information generation unit 6. FIG. 14 is an example of information generated by the preference information generation unit 6. In FIG. 14, “ID” is information for identifying a learner, “Wandering option” is an option determined by the visual recognition object determination unit 5 to be visually recognized by the learner, and is “not lost”. The “option” is an option that has been determined that the learner has not viewed so much. The “estimated understanding level” is a result of comprehensively determining “a lost option” and “an unambiguous option” and indicating the learner's level of understanding in five levels. In this example, there are four answer choices a to d, and the correct answer is a.

  The estimated understanding levels of the learners A to D will be described below.

  Since the learner A also visually recognized the options b and c in addition to the correct option a, it can be said that the learner A was lost in the options a, b and c. Conversely, since the learner A did not visually recognize the option d so much, it can be said that there is a high possibility that the learner A has read the option d and removed it from the candidates for the correct answer. Accordingly, the preference information generation unit 6 determines that the learner A has the understanding level enough to exclude the option d from the candidates for the correct answer, and generates the estimated understanding level "3" as the medium understanding level.

  The learner B visually recognizes the alternatives a and b, which are correct answers, and has not seen the alternatives c and d so much. In that one of the options a and b is narrowed down to the correct answer, the preference information generation unit 6 sets the estimated understanding degree of the learner B to “4” which is higher than that of the learner A.

  Learner C read the question sentence and options, and immediately answered without hesitation. From this, the preference information generation unit 6 determines that the understanding degree of the learner C is high, and generates the estimated understanding degree “5”.

  Since the learner D is visually recognizing the options b, c, and d other than the option a, the learner D does not include the option a that is the correct answer as a candidate for the correct answer from the beginning, and can be said to be lost in the options b, c, and d. . From this, the preference information generation unit 6 determines that the understanding degree of the learner D is not so high, and generates the estimated understanding degree “2”.

  It can be said that the learner E visually recognizes all the options in the same manner and is lost in all the options. From this, the preference information generating unit 6 determines that the understanding degree of the learner E is not so high, and generates the estimated understanding degree “1”.

(Program execution unit 2-1)
The program execution unit 30 may output display data including an object that visualizes the estimated understanding level generated by the preference information generation unit 6 to an external display device. A graph is an example of the display data.

  FIG. 15 is a bar graph showing the estimated understanding level of a certain learner who answered five question sentences. The visualization method is not limited, and a radar chart or the like may be used. Although FIG. 15 shows the estimated understanding level of one learner, the average estimated understanding level of a plurality of learners who solved the same problem may also be displayed.

  This makes it possible to visualize the estimated understanding level of the learner.

  In the present embodiment, the form of the preference information generated by the preference information generation unit 6 is not particularly limited. As shown in FIG. 14, the information may indicate the estimated understanding of a plurality of learners in each question, or may indicate the estimated understanding of a plurality of answered questions in each learner as shown in FIG. It may be information.

  For example, the information processing apparatus according to the present embodiment may further determine the following items and generate the estimated understanding level.

  In the case where the answer was shorter than the minimum threshold time required to read the entire question sentence and the answer option, the learner answered the question and option sentence without reading it in the first place. It is highly likely that it is not an answer after understanding. In this case, the preference information generation unit 6 may perform processing such as lowering the estimated understanding level.

  In addition, after weighting the answer options, when the learner is lost in a plurality of options as answer candidates, the weight of the options may be reflected in the estimated understanding level. For example, when an answer option that cannot be normally selected by a learner who should have learned in advance is set as an answer candidate, the preference information generation unit 6 may perform processing such as lowering the estimated understanding level.

(Program execution unit 2-2)
The program execution unit 30 proposes proposal information to the user based on the preference information acquired from the preference information generation unit 6. In the present embodiment, for example, the suggestion information causes the learner to review a question having a low estimated understanding level among a plurality of questions answered by each learner, which are included in the preference information generated by the preference information generation unit 6. Information. The information for reviewing is, specifically, an explanation or the like for making the learner understand the problem with a low estimated understanding level. More specifically, the information for reviewing is a message such as "It seems that the understanding of the past completed form of English grammar is low." And "Review the textbook pages 10 to 13 again." There may be.

  As a method of suggesting proposal information, for example, after the learner has completed a plurality of predetermined questions, the program execution unit 30 may display the proposal information on the display unit on a screen for confirming correctness of the answer. In addition, the program execution unit 30 may transmit the generated proposal information to the learner using communication means such as an email, a social network service (SNS), and an instant message (IM).

  In this way, the program execution unit 30 proposes the proposal information to the learner, so that the learner can enjoy the information based on his / her estimated understanding level.

  The program execution unit 30 may determine whether or not the user has visually recognized the proposal information. The determination as to whether or not the user has visually recognized the suggestion information may be made by, for example, determining whether or not the learner has visually recognized a screen displaying information for reviewing the problem, which has been suggested to the learner, by using the program execution unit 30. May be determined by reference. Information on whether or not the learner has visually recognized the information is acquired from, for example, the visually recognized object determination unit 5. The visual recognition object determination unit 5 may perform the processing described in (Visual recognition object determination unit) of the first embodiment, for example.

  In addition, the program execution unit 30 may perform the evaluation based on the result of the above-described determination. Here, the evaluation is, for example, an evaluation of whether information (suggestion information) for reviewing a problem, which is proposed to a learner, is appropriate. As an example, the program execution unit 30 evaluates by referring to information such as whether the learner who has visually recognized the information for reviewing the problem has logged in to the site for solving the problem again or has actually solved the problem. I do. As an example, the evaluation value may be a ratio of the number of learners who solved the problem again to the total number of learners who proposed information for reviewing the problem.

(Modification 2-1)
The information processing device 100 may further include a selection information acquisition unit (not shown). The selection information acquisition unit acquires the user's selection information. In the present embodiment, the selection information is the number of the option selected by the learner.

  The information processing apparatus according to the present modification includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visually recognizable object determination unit 5, a selection information acquisition unit, and an interest information generation unit 6. Have.

  The functions of the user information acquisition unit 1, the face information acquisition unit 2, the state detection unit 4, and the visible object determination unit 5 are the same as the functions of the members described in the second embodiment, and a description thereof will not be repeated.

  The taste information generation unit 6 refers to the user information acquired from the user information acquisition unit 1, the determination result acquired from the visual recognition object determination unit 5, and the selection information acquired from the selection information acquisition unit, and estimates the learner's estimated understanding level. Generate As an example, for the object that the visual recognition object determination unit 5 has determined that the learner has turned his or her eyes many times, that is, for the answer option that has been read back many times, the taste information generation unit 6 determines that the learner has a correct answer. Is determined to be a lost option as a candidate, and an estimated understanding level is generated.

  FIG. 16 is an example of information generated by the preference information generation unit 6. In FIG. 16, “ID” is information for identifying the learner, “selection number” is the number of the option actually selected by the learner of each ID, and “correct” is the correctness of the answer. “A lost option” is an option determined by the visual recognition object determination unit 5 that the learner was visually recognizable, and “an option that was not lost” was determined that the learner was not visually recognizable. Is an option. The “estimated understanding level” is a result of comprehensively determining “right or wrong”, “decided option” and “undecided option”, and indicating the level of understanding of the learner in five levels. In this example, there are four answer choices a to d, and the correct answer is a.

  In FIG. 16, the estimated comprehension level of learners A, C, and D who answered the question correctly will be described. The learners A, C, and D have all selected the correct option a, but the process of selecting the option a is different.

  Since the learner A saw the options b and c in addition to the option a, it can be said that the learner A was lost in the options a, b and c. Conversely, since the learner A did not visually recognize the option d so much, it can be said that there is a high possibility that the learner A has read the option d and removed it from the candidates for the correct answer. Accordingly, the preference information generation unit 6 determines that the learner A has the understanding level enough to exclude the option d from the candidates for the correct answer, and generates the estimated understanding level "3" as the medium understanding level.

  After reading the question sentence and the options, the learner C visually recognizes only the option a and selects the option a as the correct answer. Accordingly, the preference information generating unit 6 determines that the learner C has a high degree of understanding, and generates the estimated degree of understanding “5”.

  Since the learner D also visually recognized the options b, c, and d in addition to the option a, it can be said that the learner A was at a loss for all the options. From this, the learner D finally selected the correct option, but since all options were lost, the possibility that the correct answer was accidentally high is high, and the understanding information is not so high. Judgment is made and an estimated understanding level “2” is generated.

  In addition, the estimated comprehension levels of the learners B and E, who have incorrect answers to the question, are also determined in the same manner. For example, although the learner B was incorrect, he was confused by the choices a and b and was not confused by the other choices. Is determined, and an estimated understanding level “4” is generated. In addition, since the learner E is at a loss for all the options and is eventually incorrect, the preference information generating unit 6 determines that the understanding level is low, and generates the estimated understanding level “1”.

  Also in the present modified example, as described in <Embodiment 2>, the program execution unit 30 may propose proposal information based on the estimated understanding level and perform evaluation based on the estimated understanding level and the proposed information. Good.

  The evaluation here is an evaluation of whether the information (suggestion information) suggested to the learner for reviewing the problem was appropriate, as in <Embodiment 2>. In this modification, when the learner who has visually recognized the information for reviewing the problem solves the problem again, the evaluation is performed by further considering information such as whether or not the student's performance has improved. You may.

<Embodiment 3>
The information processing apparatus 100 according to the present embodiment shares interest information with another terminal device. FIG. 17 is a block diagram illustrating components of the information processing apparatus 100. As shown in FIG. 17, the information processing apparatus 100 according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, an interest information generation unit 6, a program execution unit. 30, an interest sharing request transmission unit 20, an interest sharing request reception unit 21, an interest sharing request approval unit 22, and an interest sharing unit 23. The functions of members other than the taste sharing request transmitting unit 20, the taste sharing request receiving unit 21, the taste sharing request approving unit 22, and the taste sharing unit 23 are the same as the functions of the members described in the first embodiment. Is omitted.

  FIG. 18 is a diagram illustrating an outline of the information processing system according to the present embodiment. As illustrated in FIG. 18, the information processing system according to the present embodiment includes a server 300 and a plurality of terminal devices (10, 10a) as an example of a plurality of information processing devices 100.

  The preference sharing request transmission unit 20 transmits information for requesting sharing of preference information to another terminal via the server 300.

  The preference sharing request receiving unit 21 receives, from the preference sharing request transmitting unit of another terminal via the server 300, information indicating that the sharing of the preference information has been requested. The preference sharing request receiving unit 21 notifies the preference sharing request approval unit of the received information.

  Based on the information received from the preference sharing request receiving unit 21, the preference sharing request approval unit 22 displays on the display unit a notification requesting permission from the user as to whether the preference information may be transmitted to another terminal. Based on this display, the user performs an operation input indicating whether or not to permit, and the preference sharing request approval unit 22 transmits the input information to the preference sharing unit 23.

  If the information received from the preference sharing request approval section 22 is information indicating authorization, the preference sharing section 23 acquires the preference information from the preference information generation section 6 and transmits the preference information to another preference via the server 300. Send to terminal device. In addition, the preference sharing unit 23 receives preference information from another terminal via the server 300, generates preference information of both terminals by combining the preference information of its own terminal device and the preference information of the other terminal, and generates the preference information. At least a part of the taste information of both is displayed on the display unit 12. Specifically, the preference information of both terminals may be displayed on the terminal device, or only the preference information from another terminal may be displayed on its own terminal device. Further, the preference information of both terminals and only the preference information from another terminal may be appropriately switched and displayed by a user operation or the like. With such a configuration, the user can know the taste information of the other party who is the user of the other terminal. In addition, it is possible to know both taste information in consideration of one's own taste information and the other party's taste information.

  The server 300 manages the information for requesting the sharing of the taste information described above in association with the terminal ID or the user ID of the source terminal device and the terminal ID or the user ID of the destination terminal device. . Further, the server 300 manages the “hobby information” in association with the terminal ID or the user ID of the terminal device from which the hobby information is obtained and the terminal ID or the user ID of the terminal device that can share the hobby information. I have.

  As an example, a case where the taste information of a restaurant is shared will be described. When the user A and the user B select a restaurant to be used by two people, the user A and the user B share the taste information regarding each other's restaurants, so that the tastes are the closest to each other or the same. A restaurant can be selected.

  For example, the taste sharing request transmission unit of the terminal device of the user A transmits information for requesting the terminal device of the user B to share the taste information of the restaurant of the user B. The taste sharing request receiving unit 21 of the terminal device of the user B notifies the information received from the taste sharing request transmitting unit of the user A to the taste sharing request approval unit 22, and displays the taste information on the display unit of the terminal device of the user B. The user B may be notified of the permission for transmission to another terminal. When the user B performs an input for approval, the preference sharing request approval unit 22 transmits information of the approval to the preference sharing unit 23. The preference sharing unit 23 acquires the preference information from the preference information generation unit 6 and transmits the preference information to the terminal device of the user A. The preference information generation unit 6 acquires preference information from the terminal device of the user A, generates preference information of the two persons based on the preference information of the user A and the user B, and displays the generated result on the display unit 12. Let it.

(Program execution unit 3)
The program execution unit 30 generates an avatar image corresponding to the user, as described in the (program execution unit 1-1) of the first embodiment, and detects the user state detected by the state detection unit 4 and the visual object determination unit 5 The avatar image according to at least one of the object visually recognized by the user, the emotion of the user, and the preference information acquired from the preference information generation unit 6 is generated.

  In the present embodiment, the program execution unit 30 may display the generated avatar image on the display unit 12 of the terminal device of the user so that the user can view the avatar image, for example. , And may be transmitted to another terminal device. That is, the avatar image may be shared with the interest information among a plurality of users.

  As described above, the program executing unit 30 generates the avatar image, so that the object visually recognized by the user, the emotion of the user, and the interest information can be visualized. Also, by sharing the avatar image among a plurality of users, it is possible to increase the affinity between the users while watching the display unit of the terminal device.

  FIG. 19 is an example in which the taste information of the user A and the user B is displayed. For example, a store that is included in both of the preference information of the user A and the preference information of the user B is displayed at a higher rank as the preference information common to both. In FIG. 19, for example, stores Y, P, X, and C displayed in the upper row on the browser screen 16 are stores that are common preferences of the user A and the user B. Further, when the browser screen 16 is displayed on the display unit 12 of the terminal device of the user A, the display unit 12 of the terminal device of the user A may display only the store desired by the user B.

  The program execution unit 30 of the present embodiment may appropriately change the layout of the display unit 12. The processing of the program execution unit 30 is the same as the processing of the program execution unit 30 described in the first embodiment.

  The number of terminals sharing interest information is not particularly limited, and may be shared among three or more terminal devices.

  In the present embodiment, the terminal device includes the preference information generation unit 6 and the preference sharing unit 23, and the preference information is shared by a plurality of terminal devices. However, the present invention is not limited to this mode. In the example illustrated in FIG. 18, the server 300 includes the preference information generation unit 6 and the preference sharing unit 23, and a plurality of users share each other's preference information and generate preference information of a plurality of people via the server. It may be in a form.

  A configuration in which the server includes the preference information generation unit 6 will be described in a fourth embodiment.

<Embodiment 4>
[Main Configurations of Information Processing Device 400 and Server 200]
An information processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 20 is a block diagram illustrating components of the terminal device 10 including the information processing device 400 according to the present embodiment and a server 200 that is the information processing device according to the present embodiment.

  The terminal device 10 acquires the user information and the face information of the user, transmits the acquired information to the server 200, and the server 200 generates the interest information by referring to the received information. The server 200 transmits the generated preference information to the terminal device 10, and the terminal device 10 acquires the preference information.

  The information processing apparatus 400 according to the present embodiment includes: a user information acquisition unit 1 that acquires user information of a registered specific user; a face information acquisition unit 2 that acquires user face information; The information processing apparatus includes a preference information acquisition unit 24 that acquires preference information obtained by referring to face information, and a program execution unit 30. In addition, the terminal device 10 including the information processing device further includes a display unit.

  Thereby, the information processing apparatus 400 according to the present embodiment can acquire the user's preference information.

  The user information acquisition unit 1 and the face information acquisition unit 2 acquire user information and user face information, respectively, as in the first embodiment.

  The user information includes user attribute information, and specific user attribute information is the same as the attribute information described in the first embodiment.

  Further, the user information includes user identification information for identifying the user from other users, and specific identification information of the user is the same as the identification information described in the first embodiment.

  Thereby, the attribute of the user can be obtained. In addition, since the user information includes the user identification information, another user and the target user can be identified, and the taste information of the user and the user information can be linked.

  The present embodiment is different from the first embodiment in that information acquired by the user information acquisition unit 1 and the face information acquisition unit 2 is transmitted to the server 200, as shown in FIG.

  Further, the preference information acquisition unit 24 acquires the preference information from the server 200 and causes the display unit 12 to display the acquired result.

  The information processing device 400 may further include a program execution unit (not shown) that executes a program based on the preference information.

  Thereby, the information processing device 400 can execute a program based on the user's preference information.

(Program execution unit 4-1)
The program execution unit 30 generates an avatar image corresponding to the user, the avatar image corresponding to at least one of a state of the user, an object visually recognized by the user, emotion of the user, and interest information. The details of the generation of the avatar image performed by the program execution unit are as described in the first embodiment.

  As described above, the program executing unit 30 generates the avatar image, so that the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  The program execution unit 30 may propose the proposal information based on the preference information to the user, and may perform the evaluation based on the preference information and the proposal information. The details of the proposal and evaluation of the proposal information performed by the program execution unit are as described in the first embodiment.

  In this way, the program execution unit 30 proposes the proposal information to the user, so that the user can enjoy the information based on his / her own taste. In addition, the evaluation performed by the program execution unit 30 makes it possible to evaluate whether the proposal information is appropriate.

  The program executing unit may change display information to be displayed on an external display device based on the preference information. The details of the change of the display information performed by the program execution unit are as described in the first embodiment.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

[Main Configuration of Server 200]
The server 200 will be described with reference to FIG. The server 200 according to the present embodiment includes a user information acquisition unit 1a that acquires user information of a registered specific user, a face information acquisition unit 2a that acquires face information of the user, user information of the user, The system includes a taste information generation unit 6a that generates taste information about the user's taste with reference to the user's face information, and a program execution unit 30a that executes a program based on the taste information.

  Thereby, the server 200 can suitably generate the user's preference information.

  As shown in FIG. 20, the server 200 according to the present embodiment further includes a transmission unit 19 that transmits the preference information.

  The user information acquisition unit 1a and the face information acquisition unit 2a included in the server 200 receive the user information and the face information from the user information acquisition unit 1 and the face information acquisition unit 2 in the terminal device 10, respectively. The information is transmitted to the information generator 6a.

  The preference information generation unit 6a generates the user preference information by referring to the received user information and face information. The specific operation of the preference information generation unit 6a is the same as that of the preference information generation unit 6 described in the first embodiment, but the preference information generation unit 6a of the server 200 according to the present embodiment includes , A state detection unit 4a, and a visible object determination unit 5a. Here, the functions of the state detection unit 4a and the visible object determination unit 5a are the same as the functions of the state detection unit 4 and the visible object determination unit 5 described in the first embodiment.

  As described above, by including the state detection unit 4a and the visible object determination unit 5a, the server 200 can detect the state of the user and can appropriately determine the object visually recognized by the user.

  In addition, the information processing apparatus 400 that performs transmission and reception with the server 200 refers to the state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit 2 and the result detected by the state detection unit. A visually recognized object determination unit for determining the visually recognized object may be provided.

  As described above, by including the state detection unit and the visible object determination unit, the information processing apparatus 400 can detect the state of the user and can appropriately determine the object visually recognized by the user.

  The visual recognition object determination unit determines the object visually recognized by the user by further referring to the layout information of the page being viewed by the user, similarly to the visual recognition object determination unit according to the first embodiment.

  In this way, the visual object determination unit can accurately determine which object on the page the user is visually recognizing by further referring to the page layout information.

  In addition, the visible object determination unit may further determine at least one of the degree of concentration of the user and the emotion of the user with reference to a result detected by the state detection unit. The function of the visible object determination unit is the same as that of the visible object determination unit 5 according to the first embodiment.

  Thus, it is possible to determine how much the user is interested in the object and what kind of emotion the user is viewing the object.

  The information processing device 100 may further include an image acquisition unit (not shown) that acquires a captured image including the user's field of view from an imaging device worn by the user. Further, the visible object determination unit may determine the object visually recognized by the user with reference to the captured image acquired by the image acquisition unit and the result detected by the state detection unit.

  The functions of the image acquisition unit and the visible object determination unit are the same as the functions of the image acquisition unit 15 and the visible object determination unit 5 described in the first embodiment.

  This makes it possible to appropriately determine the object visually recognized by the user.

  The state of the user detected by the state detection unit is the same as that of the state detection unit 4 according to the first embodiment, and the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, At least one of nose movement, lip movement and chin movement.

  Thereby, the degree of concentration of the user can be suitably determined.

  Here, similar to the first embodiment, the interest information is information on an object having a strong interest of the user and an object having a weak interest.

  Thereby, the preference information generation unit 6a included in the server 200 can know information of the object that each user has a strong interest and the object that the user is not interested in, and the preference information generation unit generates the weighted preference information. be able to. In addition, the preference information acquisition unit 24 included in the information processing apparatus 400 acquires the preference information, so that the user can acquire the preference information based on his / her own preference.

  The information processing apparatus is a server, and the form in which the preference information is generated by the preference information generation unit provided in the server is not limited to the present embodiment, and may be applied to the above-described first to third embodiments.

  As described above, by providing the taste information generation unit in the server, for example, even if the terminal device used by the user is not always the same, it is possible to generate the taste information of the user on the server. As long as the user information and face information of the user can be obtained, even if the user uses any terminal device such as a personal computer, a tablet, and a smartphone, it is possible to stably generate the interest information of the user.

  Providing the server with the preference information generation unit is particularly useful in the form of sharing the preference information of a plurality of users described in the third embodiment. When the preference information of a plurality of users is shared, for example, the server may further include a preference sharing unit. With such a configuration, it is possible to preferably generate taste information of a plurality of users on the server.

(Program execution unit 4-2)
The program execution unit 30a generates an avatar image corresponding to the user, the avatar image corresponding to at least one of the state of the user, the object visually recognized by the user, the emotion of the user, and interest information. The processing performed by the program execution unit 30a is the same as the processing described in (Program execution unit 1-1) of the first embodiment.

  Thereby, the object visually recognized by the user, the emotion of the user, and the interest information can be visualized.

  The avatar image generated by the program execution unit 30a according to the present embodiment is transmitted to the preference information acquisition unit 24 of the terminal device 10 via the transmission unit 19. In FIG. 20, the mode is transmitted only to the terminal apparatus 10, but the type and number of destination apparatuses are not particularly limited, and may be transmitted to a plurality of apparatuses.

  As described above, the server transmits the avatar image generated by the program execution unit 30a to a plurality of devices, so that a plurality of users can view the avatar image.

  FIG. 20 shows a mode in which the face information and the user information are received only from the terminal device 10. However, the type and number of the devices that transmit the face information and the user information are not particularly limited. Face information and user information may be received from a plurality of devices. In this case, the preference information generation unit 6a of the server 200 may generate the preference information of each user by referring to the face information and the user information of a plurality of users. Further, the preference information generation unit 6a may generate preference information in which preference information of a plurality of users is integrated by referring to face information and user information of a plurality of users. The program execution unit 30a may generate an avatar image corresponding to each user with reference to the preference information of each user generated by the preference information generation unit 6a. Further, the program execution unit 30a may be configured to be able to transmit an avatar image corresponding to a certain user to a terminal device associated with another user. As an example, the program execution unit 30a generates an image for displaying avatar images corresponding to a plurality of users in one virtual space, and transmits the image to each terminal device associated with the plurality of users. A possible configuration may be adopted. More specifically, the program execution unit 30a arranges an avatar image corresponding to each user in a virtual space that simulates a real store, and displays an image for making it appear that each user is shopping at the real store. May be generated.

  As described above, the server receives the face information and the user information of the plurality of users from the plurality of devices, so that the preference information generation unit 6a can generate the preference information of the plurality of users. Further, the program execution unit 30a can generate an avatar image with reference to the preference information of a plurality of users.

  The program execution unit 30a may perform evaluation based on the preference information generated by the preference information generation unit 6a and the proposal information proposed by the program execution unit 30a. The details of the proposal and evaluation of the proposal information performed by the program execution unit 30a are as described in the first embodiment.

  As described above, the program execution unit 30a proposes the proposal information to the user, so that the user can enjoy information based on his / her own taste. In addition, the evaluation performed by the program execution unit 30a makes it possible to evaluate whether the proposal information is appropriate.

  The program execution unit 30a may change display information to be displayed on an external display device based on the preference information. The details of the change of the display information performed by the program execution unit 30 are as described in the first embodiment.

  As a result, a layout or the like based on the taste information of each user can be provided to each user.

[Example of software implementation]
The control block of the information processing device 100 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

  In the latter case, the information processing apparatus 100 includes a computer that executes instructions of a program that is software for realizing each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium storing the program. Then, in the computer, the object of the present invention is achieved when the processor reads the program from the recording medium and executes the program. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit can be used. Further, a RAM (Random Access Memory) for expanding the program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (a communication network, a broadcast wave, or the like) capable of transmitting the program. Note that one embodiment of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

  The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the computer is operated as each unit (software element) included in the information processing device, so that the information processing device is connected to the computer. The present invention also includes a control program for an information processing apparatus to be realized by a computer and a computer-readable recording medium on which the control program is recorded.

  The present invention is not limited to the embodiments described above, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Further, new technical features can be formed by combining the technical means disclosed in each embodiment.

1, 1a User information acquisition section 2, 2a Face information acquisition section 4, 4a State detection section 5, 5a Visual object determination section 6, 6a Preference information generation section 15 Image acquisition section 30, 30a Program execution section 24 Preference information acquisition section 100 , 400 Information processing device

Claims (44)

A user information acquisition unit that acquires user information of a registered specific user,
A face information acquisition unit that acquires the user's face information;
A state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit;
A visible object determination unit that determines an object visually recognized by the user with reference to a result detected by the state detection unit;
An understanding level information generating unit configured to generate an estimated understanding level of the user regarding learning with reference to a state of the user by the state detecting unit and a determination result by the visible object determining unit;
An information processing apparatus, comprising: a program execution unit that executes a program based on the estimated understanding level.   The information processing device according to claim 1, wherein the estimated understanding level is an estimated understanding level of the user regarding the selection problem.   The information processing apparatus according to claim 2, wherein the estimated understanding level includes a number of an answer selected by the user.   The information processing device according to claim 3, wherein the understanding level information generation unit generates the estimated understanding level by referring to the correctness of the number of the answer finally selected by the user. The visual object determination unit classifies the object into one of a lost option and a non-lost option according to a degree of the user's visibility of the object,
The said understanding level information production | generation part produces | generates the said estimated understanding level further with reference to at least one selected from the group which consists of right or wrong of the answer of the user, the option which was lost, and the option which was not lost. 5. The information processing apparatus according to claim 4. The visual recognition object determination unit includes:
The information processing apparatus according to any one of claims 1 to 5, wherein an object visually recognized by the user is determined by further referring to layout information of a page viewed by the user. . The visual recognition object determination unit includes:
The at least one of the degree of concentration of the user and the emotion of the user is further determined by referring to a result detected by the state detection unit, The method according to any one of claims 1 to 6, wherein Information processing device. From the imaging device worn by the user, further comprising an image acquisition unit that acquires a captured image including the user's field of view,
The visual recognition object determination unit includes:
A captured image obtained by the image obtaining unit;
A result detected by the state detection unit,
The information processing apparatus according to claim 1, wherein the information processing apparatus determines an object visually recognized by the user with reference to the reference. The program execution unit includes:
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the state of the user, the object visually recognized by the user, the emotion of the user, and the estimated understanding level. The information processing apparatus according to claim 1, wherein: The program execution unit includes:
The information processing apparatus according to claim 1, wherein proposal information based on the estimated understanding level is proposed to the user. The program execution unit includes:
The information processing apparatus according to claim 10, wherein an evaluation is performed based on the estimated understanding level and the proposal information. The program execution unit includes:
The information processing apparatus according to claim 1, wherein the estimated understanding level is output as display data to an external display device. The program execution unit includes:
13. The information processing apparatus according to claim 1, wherein display data including an object in which the estimated understanding level is visualized is output to an external display device. The program execution unit includes:
14. The information processing apparatus according to claim 1, wherein display information to be displayed on an external display device is changed based on the estimated understanding level.   The information processing apparatus according to claim 1, wherein the user information includes attribute information of the user.   The information processing apparatus according to claim 1, wherein the user information includes user identification information for identifying the user from another user.   The user's state detected by the state detection unit includes the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement and chin movement. The information processing apparatus according to claim 1, wherein the information processing apparatus is at least one of them. An information processing program for causing a computer to function as the information processing apparatus according to claim 1,
An information processing program for causing a computer to function as the user information acquisition unit, the face information acquisition unit, the state detection unit, the visible object determination unit, the understanding level information generation unit, and the program execution unit. A user information acquisition step of acquiring user information of a registered specific user,
A face information acquisition step of acquiring face information of the user;
A state detection step of detecting the state of the user from the face information acquired in the face information acquisition step,
A visible object determining step of determining an object visually recognized by the user with reference to a result detected in the state detecting step;
An understanding level information generating step of generating an estimated understanding level of the user regarding learning with reference to the user information of the user and a determination result in the visual object determining step;
A program execution step of executing a program based on the estimated understanding level. A user information acquisition unit that acquires user information of a registered specific user,
A face information acquisition unit that acquires the user's face information;
An understanding level information acquisition unit configured to acquire the user's estimated understanding level regarding learning, which is obtained by referring to the user information of the user and the face information of the user,
An information processing apparatus comprising:   21. The information processing apparatus according to claim 20, wherein the estimated understanding level is an estimated understanding level of the user for the selection problem.   22. The information processing apparatus according to claim 21, wherein the estimated understanding level includes a number of an answer selected by the user.   23. The information processing apparatus according to claim 22, wherein the understanding level information obtaining unit obtains the estimated understanding level by referring to the correctness of the answer number finally selected by the user.   24. The information processing apparatus according to claim 20, further comprising a program execution unit that executes a program based on the estimated understanding level. The program execution unit includes:
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the state of the user, the object visually recognized by the user, the emotion of the user, and the estimated understanding level. The information processing apparatus according to claim 24, wherein The program execution unit includes:
26. The information processing apparatus according to claim 24, wherein proposal information based on the estimated understanding level is proposed to the user. The program execution unit includes:
The information processing apparatus according to claim 26, wherein an evaluation is performed based on the estimated understanding level and the proposal information. The program execution unit includes:
28. The information processing apparatus according to claim 24, wherein display information to be displayed on an external display device is changed based on the estimated understanding level. A state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit;
A visible object determination unit that determines an object visually recognized by the user with reference to a result detected by the state detection unit;
The information processing apparatus according to any one of claims 20 to 28, further comprising: The visual recognition object determination unit classifies the object into one of a lost option and a non-lost option according to the degree of visibility of the object by the user,
The method according to claim 29, wherein the understanding level information obtaining unit obtains the estimated level of understanding by further referring to at least one selected from the group consisting of correct or incorrect answer of the user, a lost option, and a clear option. An information processing apparatus according to claim 1. The visual recognition object determination unit includes:
31. The information processing apparatus according to claim 29, wherein an object visually recognized by the user is determined by further referring to layout information of a page viewed by the user. The visual recognition object determination unit includes:
The method according to any one of claims 29 to 31, wherein at least one of the degree of concentration of the user and the emotion of the user is further determined with reference to a result detected by the state detection unit. Information processing device. From the imaging device worn by the user, further comprising an image acquisition unit that acquires a captured image including the user's field of view,
The visual recognition object determination unit includes:
A captured image obtained by the image obtaining unit;
A result detected by the state detection unit,
The information processing apparatus according to any one of claims 29 to 32, wherein the information processing apparatus determines the object visually recognized by the user with reference to.   The user's state detected by the state detection unit includes the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement and chin movement. The information processing apparatus according to any one of claims 29 to 33, wherein the information processing apparatus is at least one of them.   The information processing apparatus according to any one of claims 29 to 34, wherein the user information includes attribute information of the user.   The information processing apparatus according to any one of claims 29 to 35, wherein the user information includes user identification information for identifying the user from other users. A user information acquisition unit that acquires user information of a registered specific user,
A face information acquisition unit that acquires the user's face information;
An understanding level information generating unit configured to generate the estimated understanding level of the user regarding learning with reference to the user information of the user and the face information of the user;
An information processing apparatus comprising: a program execution unit that executes a program based on the estimated understanding level.   The information processing device according to claim 37, wherein the estimated understanding level is an estimated understanding level of the user for the selection problem.   The information processing apparatus according to claim 38, wherein the estimated understanding level includes a number of an answer selected by the user.   40. The information processing apparatus according to claim 39, wherein the understanding level information generation unit generates the estimated understanding level by referring to the correctness of the answer number finally selected by the user. The program execution unit includes:
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the state of the user, the object visually recognized by the user, the emotion of the user, and the estimated understanding level. The information processing apparatus according to any one of claims 37 to 40, wherein the information processing apparatus is characterized in that: The program execution unit includes:
The information processing apparatus according to any one of claims 37 to 41, wherein proposal information based on the estimated understanding level is proposed to the user. The program execution unit includes:
43. The information processing apparatus according to claim 42, wherein an evaluation is performed based on the estimated understanding level and the proposal information. The program execution unit includes:
The information processing apparatus according to any one of claims 37 to 43, wherein display information to be displayed on an external display device is changed based on the estimated understanding level.

Images