JP7263825B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

2. Description of the Related Art In recent years, a method of estimating a customer's preferences and the like by collating the customer's purchasing behavior with the customer's trend has been used in marketing.

Patent Publication No. 4991440 (Registered May 11, 2012)

However, the conventional technology as described above is intended only for accumulating data on purchase histories and preferences of an unspecified number of customers (users) and using it for marketing, and provides information optimized for each user. It was not something that considered the benefit for the user, such as the one provided. In addition, since the trend of each user was not followed after the information was provided, the provided information could not be evaluated.

It is an object of one aspect of the present invention to provide each user with optimized information and to provide each user with benefits.

Another object of one aspect of the present invention is to follow trends of each user after information is provided and to evaluate the provided information.

To solve the above problems, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a specific registered user, and a face information acquisition unit that acquires face information of the user. a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit; and a visual recognition that determines an object that the user is viewing by referring to the result detected by the state detection unit. an object determination unit; a preference information generation unit that generates preference information about the user's preference by referring to the state of the user by the state detection unit and the judgment result of the visible object determination unit; and a program execution unit for executing.

Thereby, the information processing apparatus can suitably generate the user's preference information.

Further, in order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a visible object determination unit, and the visible object determination unit further obtains layout information of a page viewed by the user. By referring to it, the object visually recognized by the user is determined.

Accordingly, the information processing apparatus 100 can accurately determine the object visually recognized by the user.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a visible object determination unit, the visible object determination unit refers to the result detected by the state detection unit, At least one of a user's degree of concentration and said user's emotion is further determined.

This makes it possible to determine to what extent the user is interested in the object and with what kind of emotion the user is visually recognizing the object.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention further includes an image acquisition unit that acquires a captured image including the user's field of view from an imaging device worn by the user. A visible object determination unit determines an object visually recognized by the user by referring to the captured image acquired by the image acquisition unit and the result detected by the state detection unit.

Thereby, the object visually recognized by the user can be suitably determined.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, the program execution unit is an avatar image corresponding to the user, and the state of the user is displayed. , an avatar image corresponding to at least one of the object viewed by the user, the user's emotion, and the taste information.

This makes it possible to visualize the object visually recognized by the user, the user's emotion, and preference information.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information to the user based on the preference information.

Thereby, the user can enjoy information based on his or her taste.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs evaluation based on the preference information and the proposal information. .

This makes it possible to evaluate whether the proposed information is appropriate.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit outputs the preference information as display data to an external display device. .

This makes it possible to visualize the estimated comprehension level of the learner.

In order to solve the above problem, the preference information is the estimated degree of understanding of the user who answered a certain question.

As a result, the user's estimated degree of comprehension can be preferably generated.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit transmits display data including an object that visualizes the estimated comprehension level to an external device. Output to the display device.

This makes it possible to visualize the estimated comprehension level of the learner.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit causes an external display device to display based on the preference information. change the display information of

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes attribute information of the user.

This makes it possible to acquire user attributes.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes information identifying the user from other users. contains user identification information for

This makes it possible to identify the target user from other users, and associate the user's preference information with the user information.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit, The state of the user detected by the unit includes at least one of the user's line of sight, state of the pupil, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. is one.

Thereby, the information processing apparatus can suitably detect the state of the user.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a preference information generation unit that generates preference information about the user's preference, and the preference information generated by the preference information generation unit The information is information about objects in which the user has a strong interest and objects in which the user has a weak interest.

As a result, it is possible to know information about objects in which each user has a strong interest and information about objects in which the user has a weak interest, and the preference information generating section can generate weighted preference information.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a temporary ID assigning unit that assigns a temporary ID to the face information; and a matching unit that matches the payment information.

This allows the information processing device to temporarily distinguish face information from other face information.

In order to solve the above problem, an information processing program according to one aspect of the present invention is an information processing program for causing a computer to function as the information processing apparatus, and causes the computer to function as the preference information generating section.

Thereby, the information processing program can cause the computer to function as a preference information generation unit.

In order to solve the above problems, an information processing method according to an aspect of the present invention is an information processing method executed by an information processing apparatus, which includes user information acquisition for acquiring user information of a specific registered user. a face information acquisition step of acquiring face information of the user; a state detection step of detecting the state of the user from the face information acquired in the face information acquisition step; and a result detected in the state detection step. Then, by referring to a visible object determination step of determining an object viewed by the user, user information of the user, and a determination result of the visible object determination step, preference information relating to the user's preference is generated. It includes a preference information generation step and a program execution step of executing a program based on the preference information.

Thereby, the preference information can be preferably generated.

To solve the above problems, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a specific registered user, and a face information acquisition unit that acquires face information of the user. and a preference information acquisition unit that acquires preference information obtained by referring to the user information of the user and the face information of the user.

Thereby, the information processing apparatus can acquire the user's preference information.

In order to solve the above problem, the information processing apparatus according to one aspect of the present invention further includes a program execution unit that executes a program based on the preference information.

Thereby, it is possible to execute a program based on the user's preference information.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, the program execution unit is an avatar image corresponding to the user, and the state of the user is displayed. , an avatar image corresponding to at least one of the object viewed by the user, the user's emotion, and the taste information.

This makes it possible to visualize the object visually recognized by the user, the user's emotion, and preference information.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information to the user based on the preference information.

Thereby, the user can enjoy information based on his or her taste.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs evaluation based on the preference information and the proposal information. .

This makes it possible to evaluate whether the proposed information is appropriate.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit causes an external display device to display based on the preference information. change the display information of

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit; and a visible object determination unit that refers to the result of detection by and determines the object visually recognized by the user.

Accordingly, the information processing apparatus 400 can detect the state of the user and preferably determine the object visually recognized by the user.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a visible object determination unit that determines an object visually recognized by a user. The object visually recognized by the user is determined by further referring to the layout information of the page being browsed.

This makes it possible to accurately determine which object on the page the user is viewing.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a visible object determination unit that determines an object visually recognized by a user, and the visible object determination unit includes the state detection unit. At least one of the user's degree of concentration and the user's emotion is further determined with reference to the results detected by the unit.

This makes it possible to determine to what extent the user is interested in the object and with what kind of emotion the user is visually recognizing the object.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention further includes an image acquisition unit that acquires a captured image including the user's field of view from an imaging device worn by the user. A visible object determination unit determines an object visually recognized by the user by referring to the captured image acquired by the image acquisition unit and the result detected by the state detection unit.

Thereby, the object visually recognized by the user can be suitably determined.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a state detection unit that detects a state of a user, and the state of the user detected by the state detection unit is the state of the user. eye gaze, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement.

As a result, the degree of concentration of the user can be suitably determined.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes attribute information of the user.

This makes it possible to acquire user attributes.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information, and the user information includes information identifying the user from other users. contains user identification information for

This makes it possible to identify the target user from other users, and associate the user's preference information with the user information.

To solve the above problems, an information processing apparatus according to an aspect of the present invention includes a user information acquisition unit that acquires user information of a specific registered user, and a face information acquisition unit that acquires face information of the user. a preference information generation unit that generates preference information about the user's preference by referring to the user information of the user and the face information of the user; and a program execution unit that executes a program based on the preference information. , is equipped with

Thereby, the information processing apparatus can suitably generate the user's preference information.

Further, in order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, the program execution unit is an avatar image corresponding to the user, and the state of the user is displayed. , an avatar image corresponding to at least one of the object viewed by the user, the user's emotion, and the taste information.

This makes it possible to visualize the object visually recognized by the user, the user's emotion, and preference information.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit proposes proposal information to the user based on the preference information.

Thereby, the user can enjoy information based on his or her taste.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit performs evaluation based on the preference information and the proposal information. .

This makes it possible to evaluate whether the proposed information is appropriate.

Further, in order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes a program execution unit, and the program execution unit displays information on an external display device based on preference information. Change the displayed information.

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

In order to solve the above problem, an information processing apparatus according to an aspect of the present invention includes a preference information generation unit that generates preference information, or a preference information acquisition unit that acquires the preference information. is information about objects in which the user has a strong interest and objects in which the user has a weak interest.

As a result, it is possible to know information about objects in which each user has a strong interest and information about objects in which the user has a weak interest, and the preference information generating section can generate weighted preference information. In addition, the user can acquire preference information based on his or her own preferences by the preference information acquisition unit acquiring the preference information.

According to one aspect of the present invention, it is possible to provide benefits to each user in accordance with each user's taste.

1 is a block diagram showing components of an information processing apparatus according to Embodiments 1 and 2 of the present invention; FIG. FIG. 3 is a diagram showing a display unit of a smart phone provided with the information processing device according to Embodiments 1 and 3 of the present invention; 1 is a diagram showing a display unit of a personal computer provided with an information processing device according to Embodiments 1 and 3 of the present invention; FIG. 4 is a table showing an example of data output by a visible object determination unit of the information processing apparatus according to Embodiments 1, 3 and 4 of the present invention; 7 is a table showing an example of preference information generated by the preference information generation unit of the information processing apparatus according to the first, third and fourth embodiments of the present invention; 4 is a table showing an example of an avatar image generated by a program execution unit of the information processing device according to Embodiment 1 of the present invention; 4 is a table showing an example of data output by a program execution unit of the information processing apparatus according to Embodiments 1, 3 and 4 of the present invention; 4 is a table showing an example of data output by a program execution unit of the information processing apparatus according to Embodiments 1, 3 and 4 of the present invention; FIGS. 1A and 1B are diagrams showing a display unit of a personal computer provided with an information processing apparatus according to Embodiments 1, 3, and 4 of the present invention, showing (a) before the display is changed and (b) the display after the display is changed; FIG. is. 4 is a flow chart showing an example of the flow of processing of the information processing apparatus according to the first embodiment of the present invention; 4 is a table showing an example of data output by a program execution unit of the information processing apparatus according to Embodiment 1 of the present invention; 4 is a table showing an example of data output by a program execution unit of the information processing apparatus according to Embodiment 1 of the present invention; FIG. 9 is a diagram showing an example of a browser screen displayed by a terminal device provided with an information processing device according to Embodiment 2 of the present invention; FIG. 10 is a diagram showing an example of preference information generated by the preference generation unit of the information processing apparatus according to the second embodiment of the present invention; FIG. 10 is a diagram showing an example of preference information generated by the preference generation unit of the information processing apparatus according to the second embodiment of the present invention; FIG. 10 is a diagram showing an example of preference information generated by the preference generation unit of the information processing apparatus according to the second embodiment of the present invention; FIG. 7 is a block diagram showing components of an information processing apparatus according to Embodiment 3 of the present invention; FIG. 11 is a diagram showing an overview of transmission and reception of preference information between information processing apparatuses according to Embodiments 3 and 4 of the present invention; FIG. 10 is a diagram showing an example of a browser screen displayed by a terminal device provided with an information processing device according to Embodiment 3 of the present invention; FIG. 11 is a block diagram showing constituent elements of an information processing apparatus according to Embodiment 4 of the present invention;

<Embodiment 1>
An embodiment of the present invention will be described in detail below. If the configuration in a specific item (embodiment) below is the same as the configuration described in other items, the description may be omitted. For convenience of explanation, members having the same functions as members shown in each item are denoted by the same reference numerals, and explanations thereof are omitted as appropriate.

[Main Configuration of Information Processing Apparatus 100]
FIG. 1 is a block diagram showing components of an information processing apparatus 100. As shown in FIG. As shown in FIG. 1, an information processing apparatus 100 according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, a preference information generation unit 6, and a program execution unit. A section 30 is provided.

Thereby, the information processing apparatus 100 can suitably generate the user's preference information.

Here, the state detection unit 4 , the visible object determination unit 5 , the preference information generation unit 6 and the program execution unit 30 are implemented as one configuration of the control unit 9 provided in the information processing apparatus 100 .

The information processing device 100 is implemented in, for example, a terminal device (eg, smart phone, tablet, personal computer, television receiver, etc.) connectable to a local network or a global network. FIG. 2 schematically shows a display screen of a smartphone and line-of-sight positions on the display screen when the information processing apparatus 100 is implemented in the smartphone. FIG. 3 schematically shows a display screen of a personal computer and a line-of-sight position on the display screen when the information processing apparatus 100 is installed in the personal computer. Other configurations included in FIGS. 2 and 3 are described below.

Also, in this embodiment, a mode in which a user purchases a product using an electronic commerce (EC) service will be described.

(User information acquisition unit)
The user information acquisition unit 1 acquires, for example, user information of a specific user registered in advance in a database of a shopping site or the like, and transmits the acquired user information to the preference information generation unit 6 . As an example, the user information includes attribute information indicating user attributes and user identification information for identifying the target user from other users. User attribute information includes, for example, the user's age, gender, member type (whether the user is a paying member), and the like. User identification information is, for example, a user's ID, a user's e-mail address, and the like.

This makes it possible to acquire user attributes. In addition, since the user identification information is included in the user information, the target user can be identified from other users, and the user's preference information and the user information can be linked.

(Face information acquisition unit)
The face information acquisition unit 2 acquires the user's face information from, for example, an image acquired by an imaging unit such as a camera. The user's face information is information indicating the feature amount of the face. The facial feature amount refers to, for example, position information indicating the position of each part of the face (e.g., eyes, nose, mouth, cheeks, eyebrows, etc.), shape information indicating the shape, size information indicating the size, and the like. . In particular, the eye information is particularly useful because it is possible to evaluate the user's interest in, and the degree of interest in, the object the user is gazing at. Eye information includes, for example, end points of the inner and outer corners of the eye, edges of the iris and pupil, and the like. Further, the face information acquisition unit 2 may appropriately perform correction processing such as noise reduction and edge enhancement on the image acquired from the imaging unit. The face information acquisition section 2 transmits the extracted face information to the state detection section 4 .

(Status detector)
The state detection unit 4 detects the user's state based on the face information acquired by the face information acquisition unit 2 . After detecting the user's state, the state detection unit 4 transmits the detection result to the visible object determination unit 5 . The state of the user detected by the state detection unit 4 is, for example, the state of each part of the user's face. At least one of motion, lip motion and jaw motion.

For example, the state detection unit 4 obtains position information indicating the position of each part of the face (for example, eyes, nose, mouth, cheeks, eyebrows, etc.), which is the feature amount of the face acquired by the face information acquisition unit 2, and the shape. With reference to the shape information and the size information indicating the size, the user's state includes, for example, the user's line of sight, pupil state, number of blinks, eyebrow movement, cheek movement, eyelid movement, lip movement. At least one of movement and jaw movement may be detected.

Thereby, the information processing apparatus can suitably detect the state of the user.

For example, the sight line detection method is not particularly limited. A method of detecting the movement destination of the line of sight of the The type of the point light source is not particularly limited, and includes visible light and infrared light. For example, by using an infrared LED, it is possible to detect the line of sight without giving discomfort to the user. In line-of-sight detection, if the line-of-sight does not move for a predetermined time or longer, it can be said that the user is gazing at the same place.

A method for detecting the state of the pupil is not particularly limited, but for example, a method of detecting a circular pupil from an eye image using Hough transform can be used. In general, people tend to open their eyes when they are concentrating, so the degree of concentration of the user can be evaluated by detecting the size of the pupil. For example, the size of the pupil is detected for a predetermined period of time, and it can be said that there is a high possibility that the user is gazing at a certain target during the period in which the pupil is enlarged within the predetermined period of time. A threshold may be set for pupil size, and a pupil size greater than or equal to the threshold may be evaluated as "open", and a pupil size less than the threshold may be evaluated as "closed".

The method for detecting the number of blinks is not particularly limited, but for example, the user's eyes are irradiated with infrared light, and the difference in the reflected amount of infrared light between when the eyes are open and when the eyes are closed is detected. methods and the like. In general, when a person is concentrating, he or she tends to blink at low frequency and at stable intervals. Therefore, by detecting the number of blinks, the degree of concentration of the user can be evaluated. For example, if the number of blinks is detected for a predetermined period of time and the blinking is performed at stable intervals within the predetermined period of time, it can be said that there is a high possibility that the user is gazing at a certain target.

The state detection unit 4 detects at least one of the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. Good, but preferably a combination of these. By combining the detection methods in this way, the state detection unit 4 can suitably evaluate the degree of concentration of the user when visually recognizing a certain object.

In addition to the state of the eyes, for example, eyebrow movements such as lifting the inside of the eyebrows or raising the outside, eyelid movements such as raising the upper eyelids and tightening the eyelids, and nose movements such as wrinkling the nose. , lip movements such as lifting the upper lip and pursing the lips, cheek movements such as lifting the cheeks, and jaw movements such as lowering the chin. As the state of the user, states of a plurality of parts of the face may be combined.

In addition, when the user's facial expression is detected, the state detection unit 4 can also calculate information on the user's facial expression by machine learning. Although the specific configuration of the learning process for acquiring information about facial expressions does not limit the present embodiment, for example, any one of the following machine learning techniques or a combination thereof can be used.

・Support Vector Machine (SVM)
・Clustering
・Inductive Logic Programming (ILP)
・Genetic Algorithms (GP)
・Bayesian Network (BN)
・Neural Network (NN)
When using a neural network, the data may be preprocessed and used as input to the neural network. For such processing, techniques such as data argumentation can be used in addition to one-dimensional or multi-dimensional arraying of data.

Moreover, when using a neural network, a convolutional neural network (CNN: Convolutional Neural Network) including convolution processing may be used. More specifically, one or more layers included in the neural network include a convolution layer that performs a convolution operation, and a filter operation (product-sum operation) is performed on input data input to the layer. may be configured. Further, when performing filter calculation, processing such as padding may be used in combination, or an appropriately set stride width may be employed.

As the neural network, a multi-layered or super multi-layered neural network with tens to thousands of layers may be used.

(Visible Object Determination Unit)
The visible object determination unit 5 further refers to the layout information of the page that the user is viewing, and determines the object that the user is viewing.

Accordingly, the information processing apparatus 100 can accurately determine the object visually recognized by the user.

Specific processing of the visible object determination unit 5 will be described with reference to FIGS. 2 and 3, taking the terminal device 10 including the information processing device 100 as an example. FIG. 2 shows a smart phone provided with the information processing device 100, and FIG. 3 shows a personal computer provided with the information processing device 100. As shown in FIG.

In FIG. 2 , a terminal device (smartphone) 10 including an information processing device 100 according to this embodiment further includes an imaging unit 14 and a display unit 12 . For example, as shown in FIG. 2, it is assumed that the display area of the browser is the same size as the display unit of the smartphone, and the product images are arranged side by side. For example, the display order of products, the manufacturer of each product, and the product name can be extracted from the source code of the web page displayed by the display unit 12 . Since each product image area is controlled by the browser, the visible object determination unit 5 can refer to the source code and the browser to acquire the coordinates of the image area of each product. The visible object determination unit 5 compares the obtained coordinates of the image area of each product with the line-of-sight information obtained from the state detection unit 4, and determines whether the position coordinates of the user's line of sight are within the coordinates of the image area of which product. Determine if there is For example, as shown in FIG. 2, when it is determined that the position coordinates of the user's line of sight are within the coordinates of the image area 18 of product a, the visible object determination unit 5 determines that the position coordinates of the user's line of sight are: Start counting the time that product a stays within the coordinates of the image area. In this way, by collating the position coordinates of the user's line of sight with the coordinates of the product image area, it is possible to suitably determine which product the user is visually recognizing. In addition to the line-of-sight information, the state of the pupil, the number of blinks, the movement of the eyebrows, the movement of the eyelids, the movement of the cheeks, the movement of the nose, the movement of the lips, and the movement of the jaw, as described in (state detection section). By referring to the motion detection result, it is possible to more preferably determine which product the user is concentrating on.

In FIG. 3, a terminal device (personal computer) 10 including an information processing device 100 according to the present embodiment also further includes an imaging unit 14 and a display unit 12, as in FIG. In the case of FIG. 3 , the size of the browser screen 16 is smaller than that of the display unit 12 , so the visible object determination unit 5 also needs to acquire information on the position of the browser screen 16 on the display unit 12 . Since the position and area of the browser are controlled by the OS (Operating System) of the information processing apparatus 100, the visible object determination unit 5 refers to the OS in addition to the source code and browser to determine the coordinates of the image area of each product. can be obtained. For example, as shown in FIG. 3, when it is determined that the position coordinates of the user's line of sight are within the coordinates of the image area 18 of product b, the visible object determination unit 5 determines that the position coordinates of the user's line of sight are: Start counting the time spent within the coordinates of the image area of product b.

Further, the visible object determination unit 5 may refer to the result obtained from the state detection unit 4 and further determine at least one of the user's degree of concentration and the user's emotion.

This makes it possible to determine to what extent the user is interested in the object and with what kind of emotion the user is visually recognizing the object.

The visible object determination unit 5 determines the user's emotion from the state of each part of the user's face obtained from the state detection unit 4 . For example, the Paul Ekman emotion classification model may be used to determine the user's emotion. The emotion classification model includes, for example, "happiness", "surprise", "anger", "disgust", "fear", and "sadness". The emotion classification model may include "normal". Furthermore, the emotion classification model includes "fun", "contempt", "satisfaction", "confusion", "excitement", "guilt", "self-esteem based on achievement", "peace of mind", "satisfaction", It may also include emotions such as "shame" and composites of these emotions (eg, anger + disgust, happiness + satisfaction, etc.).

In addition, the visible object determination unit 5 determines which product the user is concentrating on viewing and determines the user's emotion using the machine learning method described in (state detection unit). good.

The visible object determination unit 5 transmits the determination result to the preference information generation unit 6 .

(Image acquisition unit)
The information processing apparatus 100 may further include an image acquisition unit 15 that acquires a captured image including the user's field of view from an imaging device worn by the user. Here, the imaging device worn by the user is not particularly limited as long as it can detect the state of the user, but examples include a camera mounted on a glasses-type wearable device and a camera mounted on a head-mounted device. These devices may be transmissive or non-transmissive. In the case of the non-transmissive type, for example, a configuration in which a camera (imaging device) and a display (display section) viewed by a user are provided in the same device may be employed.

A captured image including the user's field of view is, for example, an image obtained by capturing a scene, a display (display unit), and the like that the user is looking at.

The visible object determination unit 5 may determine the object visually recognized by the user by referring to the captured image acquired by the image acquisition unit 15 and the result detected by the state detection unit 4 . Specifically, the visible object determination unit 5 compares the coordinates of the captured image area with the line-of-sight information of the user detected by the state detection unit 4, and determines which coordinate in the captured image the position coordinates of the user's line of sight correspond to. Determine if compatible. The visible object determination unit 5 may be configured to recognize the object included in the captured image acquired by the image acquisition unit 15 using, for example, machine learning.

Thereby, the object visually recognized by the user can be suitably determined.

(Preference information generator)
The preference information generation unit 6 refers to the user information obtained from the user information acquisition unit 1 and the determination result obtained from the visible object determination unit 5 to generate preference information related to the user's preferences. Here, the preference information is information indicating to which object the user's interest is strong and to which object the interest is weak.

As a result, it is possible to know information about objects in which each user has a strong interest and information about objects in which the user has a weak interest, and the preference information generating section 6 can generate weighted preference information.

For example, the preference information generation unit 6 generates the information of the product visually recognized by the user, the result of the user's degree of concentration while viewing the product, and the result of the user's emotion, which are determined by the visible object determination unit 5. refer.

FIG. 4 shows data indicating the result of detection by the state detection unit 4, which is input to the preference information generation unit 6. As shown in FIG. FIG. 4 is an example of data about a user whose ID is A (hereinafter referred to as user A). The table in FIG. 4 shows, for each product, the time that user A visually recognized the product (“line of sight”), the number of blinks per minute (“blink”), and the opening and closing of the pupil (“state of pupil”). ing. From the table in FIG. 4 , it is determined that user A is in a highly focused state, based on the number of times he blinks and the state of his pupils, while he is visually recognizing product a for a long period of time. Further, from the table of FIG. 4, it is determined that user A has a short visual recognition time for product c and is not in a highly concentrated state.

The table in FIG. 5 is an example of preference information generated for each user's attributes, products with strong interest, and products with weak interest. Here, the attribute of each user is gender as an example. The preference information generator 6 generates preference information as shown in FIG. 5 by referring to the data in the table of FIG. The table in FIG. 5 indicates that user A is male, and that user A actually has a strong interest in product a and little interest in product b. Such preference information may be information indicating the preference of each user, or may be information indicating the preferences of a plurality of users as shown in FIG.

Further, the preference information generation unit 6 may generate the preference information using the machine learning technique described in (state detection unit).

In addition, the preference information generation unit 6 may be configured to generate user preference information by referring to other information such as click information together with or instead of the various information described above. Here, the click information may include information such as the position clicked or touched by the user, the number of clicks or touches, and the time interval between double clicks. Further, a configuration may be adopted in which an intensity detection unit that detects the intensity of a click or touch is provided, and the preference information generation unit 6 refers to the intensity of the click or touch to generate user preference information.

Note that when the display unit is configured as a touch panel, the strength of the user's touch can be detected by adopting a so-called resistive film method.

The preference information generator 6 transmits the generated preference information to the program execution unit 30 .

(Program execution unit 1-1)
The program execution unit 30 generates an avatar image corresponding to the user, the user's state detected by the state detection unit 4, the object visually recognized by the user determined by the visible object determination unit 5, the user's emotion, and preference information. An avatar image is generated according to at least one piece of preference information acquired from the generation unit 6 .

An avatar image is information that symbolically represents a user, and is mainly an image that reminds the user. As an example, when a user is selecting products at a shopping site, an avatar image is generated on an image that imitates a physical store.

This makes it possible to visualize the object visually recognized by the user, the user's emotion, and preference information.

The state of the user is the state of each part of the user's face described in (state detector). The object visually recognized by the user and the user's emotion are determined by the method described in (Visualized Object Determination Unit). The preference information, as described in (preference information generating unit), is information indicating which object the user is strongly interested in and which object it is weakly interested in.

FIG. 6 is an example of an avatar image generated by the program execution unit 30. As shown in FIG. FIG. 6(a) is an example of a conventional avatar image, FIG. 6(b) is an example of an avatar image corresponding to the user's expression, and FIG. 6(c) is an example of an avatar image corresponding to the user's expression and line of sight. . As shown in FIG. 6A, the conventional avatar image does not reflect the user's facial expression. The program execution unit 30 of this embodiment generates an avatar image according to the user's expression, as shown in FIG. 6(b). For example, when the user is looking at a certain product and has an expression of raising the corners of the mouth, an avatar image of the expression of raising the corners of the mouth is generated. Further, the program execution unit 30 may display a gauge that visualizes the user's degree of favor (preference) for a certain product by referring to the preference information. As shown in FIG. 6(c), an avatar image may be generated according to information on the user's line of sight. The line-of-sight information is, for example, the direction of the line of sight, the state of the pupil, the number of blinks, and the like. For example, when the user is visually recognizing an image of product a on a shopping site, the avatar looks in the direction of product a displayed on the image where products are displayed like in a real store. You may set the line of sight of the avatar image. Alternatively, the program execution unit 30 may generate an image in which the object visually recognized by the user is held by the avatar, for example, and the product held by the avatar is visually recognized.

Note that the avatar image may be displayed on the display unit 12 of the terminal device of the user so that the user can see the avatar image, or the avatar image of the user can be displayed so that other users can see the avatar image of the user. It may be transmitted to another terminal device, a server of a shopping site, or the like.

As a result, the user can objectively view the appearance of himself/herself shopping at the shopping site as if he or she were shopping at the actual store. Also, other users (store staff, etc.) can perceive the user who is shopping at the shopping site as one of the customers who are shopping at the actual store through the avatar image, so that better service can be provided to the user. It is possible to improve awareness for providing.

(Program execution unit 1-2)
The program execution unit 30 may output the preference information generated by the preference information generation unit 6 to an external display device as display data. The type of display data is not particularly limited, but includes, for example, numerical values, characters, tables, graphs, animations, and the like. An example of the table is a table such as that shown in FIG. Further, the preference information to be output may be the preference information of one specific user or may be the preference information of a plurality of users.

This makes it possible to visualize the user's preference information.

(Program execution unit 1-3)
The program execution unit 30 proposes to the user proposal information based on the preference information acquired from the preference information generation unit 6 . The proposal information is, for example, an electronic announcement of a product in which the user has a strong interest in the user's preference information, price drop information, an electronic coupon, or the like. The program execution unit 30 may transmit the generated proposal information to the user, for example, using communication means such as email, social network service (SNS), and instant message (IM). Moreover, you may notify a user as a banner advertisement.

FIG. 7 is a table showing an example of proposal information generated by the program execution unit 30 based on the preference information acquired from the preference information generation unit 6. As shown in FIG. As an example, the preference information generation unit 6 presumes that users A, C, and D have a strong interest in product a, and users E and F presumes that they have a strong interest in product b. Generate. In this case, the program execution unit 30 transmits an advertisement (suggestion information) for product a to users A, C, and D, and an advertisement for product b to users E and F, based on the preference information.

In this way, the program execution unit 30 proposes the proposal information to the user, so that the user can enjoy information based on his or her taste.

The program execution unit 30 may determine whether or not the user has viewed the suggested information. The determination of whether or not the advertisement has been viewed may be made, for example, by the program execution unit 30 referring to information on whether or not the user has accessed the address of the shopping site described in the advertisement. Further, for example, the program execution unit 30 may refer to information as to whether or not the user has logged into the shopping site within a predetermined period of time after the advertisement is distributed. Information as to whether or not the user has logged in to the shopping site can be obtained from the user information obtaining unit 1, for example. When the user logs in to the shopping site, the program execution unit 30 refers to information about which product image the user has viewed, thereby determining whether the user views the same product image as the product described in the advertisement. You can judge. Information about which product image the user has visually recognized may be determined by, for example, the visible object determination unit 5 using the method described in (Visualized object determination unit).

Also, the program execution unit 30 may perform evaluation based on the preference information generated by the preference information generation unit 6 and the proposal information proposed by the program execution unit 30 . The evaluation method will be explained below. Here, the evaluation is, for example, an evaluation as to whether or not the proposal information proposed to the user was appropriate.

FIG. 8 is a table showing an example of evaluation performed by the program execution unit 30. As shown in FIG. The table in FIG. 8 shows that when the program execution unit 30 sends an advertisement for product a to users A, C, and D, and an advertisement for product b to users E and F, each user sends an advertisement product image. This is the result of evaluation based on the result of determination as to whether or not the object was visually recognized. In the present embodiment, the evaluation performed by the program execution unit 30 may be, for example, the ratio of the number of users who viewed the advertisement to the total number of users who transmitted the advertisement as the evaluation value. Here, the evaluation value is expressed as an "attention rate" as an example. For example, among users A, C, and D who sent an advertisement for product a, the program execution unit 30 determined that users A and C had viewed the advertisement. is 66%. In addition, among the users E and F who sent the advertisement for the product b, the program execution unit 30 determined that only the user F had viewed the advertisement. It is evaluated as

In this way, evaluation by the program execution unit 30 makes it possible to evaluate whether the proposal information is appropriate.

(Program execution unit 1-4)
Based on the preference information acquired from the preference information generation unit 6, the program execution unit 30 changes display information to be displayed on an external display device. Here, the external display device is, for example, the display unit 12 of the terminal device. The display information includes, for example, image information indicating an image displayed on the display unit 12, layout information indicating coordinates of the image, and the like. Display information can be obtained from the source code, browser, and OS. Further, the change of information is not particularly limited, but as an example, it is a change of the layout of the product image to be displayed on the display unit 12 . The program execution unit 30 transmits the changed display information to the display unit 12 .

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

A specific example of processing performed by the program execution unit 30 will be described. FIG. 9 shows the display section 12 of the personal computer. FIG. 9(a) shows the display section 12 before the display is changed, and the display section 12 displays images of products a to f. FIG. 9(b) shows the display section 12 after the display has been changed.

Here, for example, it is assumed that the preference information generation unit 6 has generated preference information indicating that among the products a to f, products d, e, and f are of strong interest to the user. Based on the display information currently displayed in FIG. The display information whose layout has been changed is generated by exchanging the positions of . Based on the changed display information acquired from the program execution unit 30, the display unit 12 displays as shown in FIG. 9B. That is, the program execution unit 30 arranges and displays the images of the products d, e, and f at the most conspicuous places in the displayed screen. Note that the layout change is not limited to this form, and may include up and down replacement of the product image, left and right replacement, and the like. Further, based on the preference information of the preference information generation unit 6, the images may be rearranged one by one in descending order of interest of the user.

The timing at which the display information on the display unit 12 is changed is not particularly limited, but may be the timing at which the web page is updated, or may be changed at predetermined time intervals. Alternatively, the information may be changed when the user logs out of a shopping site or the like once and then logs in to the same site.

In the above description, as the display information change processing by the program execution unit 30, the change of the layout information included in the display information was taken as an example, but the present embodiment is not limited to this. Another example of changing display information is a configuration in which an image of a product in which the user has a strong interest is highlighted. Specifically, as an example of highlighting,
- Enclose the image of the product in a frame - Change the background color around the image of the product - Display the image of the product in a larger size than other products - Display the image of the product in a pop-up window, etc. mentioned.

(control part)
The control unit 9 comprehensively controls each unit of the information processing apparatus 100 . The control unit 9 includes a state detection unit 4 , a visible object determination unit 5 , a preference information generation unit 6 and a program execution unit 30 . Each function of the control unit 9 and all functions included in the information processing apparatus 100 may be realized by the CPU executing a program stored in a storage unit (not shown) or the like, for example.

[Processing example 1 of information processing device]
Next, processing of the information processing apparatus 100 will be described based on the flowchart of FIG.

First, use of the information processing apparatus is started (step S100) to start processing. The process proceeds to step S (hereinafter, “step” is omitted) 102 .

In S102, the user information acquisition unit 1 acquires user information (user information acquisition step). The details of the processing are as described in (User Information Acquisition Unit). The user information acquisition unit 1 transmits the user information to the preference information generation unit 6, and proceeds to S104.

In S104, the face information acquisition unit 2 acquires the user's face image from the imaging unit, and extracts each part of the face from the face image (face information acquisition step). The details of the processing are as described in (Face Information Acquisition Unit). The face information acquisition unit 2 transmits the extracted result to the state detection unit 4, and proceeds to S106.

In S106, the state detection unit 4 detects the state of the user's line of sight, blinks, pupils, and the state of each part of the face (state detection step). The details of the processing are as described in (state detector). The detection result is transmitted to the visible object determination unit 5, and the process proceeds to S108.

In S108, the visible object determination unit 5 determines the product image (visualized object) visually recognized by the user, the degree of concentration during the viewing, and the emotion during the viewing (visualized object determination step). The details of the process are as described in (Viewable Object Determination Unit). The visible object determination unit 5 transmits the determination result to the preference information generation unit 6, and proceeds to S110.

In S110, the preference information generation unit 6 refers to the user information obtained from the user information acquisition unit 1 and the determination result obtained from the visible object determination unit 5, and generates preference information (a preference information generation step). , to S112. The details of the processing are as described in (Preference Information Generation Unit).

In S112, the control unit 9 records the preference information generated by the preference information generation unit 6 in a database or the like, and proceeds to S114.

In S114, the program execution unit 30 performs at least one of generation of an avatar image, proposal of proposal information, evaluation, and change of display information (program execution step). The details of the processing performed by the program execution unit 30 are as described in (program execution unit 1-1) to (program execution unit 1-3). Proceeding to S116, the preference estimation process ends.

Thereby, the preference information can be preferably generated.

(Modification 1-1)
The information processing device 100 may further include a payment information acquisition unit (not shown). The payment information acquisition unit, for example, acquires payment information as to whether or not the user has purchased the product. The payment information can be acquired from, for example, credit card information registered in advance in a shopping site or the like, a payment server of the shopping site, or a payment server of a payment service provider. The payment information acquisition unit can acquire information about which product the user has purchased within the shopping site. Payment information is not limited to credit card information, and can be obtained from a medium linked to the user's personal information. Examples include debit card information, prepaid card information, and the like. The payment information acquisition unit transmits the acquired payment information to the preference information generation unit 6 .

Payment information is information as to whether or not a product has been purchased in shopping or the like, but it is not limited to this, and more generally speaking, it is decision-making information that reflects the intention of the user.

With such a configuration, the information processing apparatus 100 can generate more accurate preference information.

In addition, since the information processing apparatus 100 includes the payment information acquisition unit, the program execution unit 30 determines whether or not the user has purchased the product based on the proposal information, and performs evaluation based on the result of the determination. be able to.

FIG. 11 shows that when the program execution unit 30 transmits an advertisement (suggestion information) for product a to users A, C, and D, and an advertisement for product b to users E and F, each user receives an advertisement for the product. This is the result of evaluating whether or not the customer purchased the product. The closing rate in the table of FIG. 11 is the ratio of the number of users who actually purchased the advertised product to the number of users to whom the program execution unit 30 sent the advertisement. Since all users A, C, and D who sent the advertisement for product a have purchased product a, program execution unit 30 evaluates that the contract rate is 100%. As for users E and F who have sent advertisements for product b, only user E has purchased product b, so program execution unit 30 evaluates that the contract rate is 50%. At this time, it is not necessarily determined whether the user to whom the advertisement was sent actually saw the advertisement. In particular, advertisements that can be checked even in an offline state, such as e-mails, do not need to be judged.

Further, the program execution unit 30 may perform evaluation by referring to viewing information of the advertisement, ie, whether or not the user to whom the advertisement was sent actually saw the advertisement.

FIG. 12 shows that when program execution unit 30 sends an advertisement for product a to users A, C, and D, and an advertisement for product b to users E and F, each user purchases the advertised product. FIG. 10 is a diagram showing the result of evaluating whether or not The closing rate in the table of FIG. 12 is the ratio of the number of users who saw the advertisement and purchased the product to the number of users to whom the program execution unit 30 sent the advertisement. As shown in the table of FIG. 12, users A, C, and D who sent an advertisement for product a all purchased product a, but only users A and C saw the advertisement and purchased product a. Therefore, the program execution unit 30 evaluates that the contract rate is 66%. For users E and F who sent an advertisement for product b, none of the users saw the advertisement and ended up purchasing the product.

Regarding the viewing information of the advertisement, it may be determined whether or not the advertisement is viewed using the method described in (Viewing Object Determining Unit).

(Modification 1-2)
The information processing apparatus 100 according to this modified example includes the same units as the information processing apparatus 100 according to the first embodiment, and processes performed by the respective units are the same except for the visible object determination unit 5 . The visible object determination unit 5 will be described later. In addition, the information processing apparatus 100 according to the present embodiment includes a payment information acquisition unit (not shown) that acquires user payment information, a temporary ID assignment unit (not shown) that assigns a temporary ID, a temporary ID and the user The information processing apparatus 100 differs from the information processing apparatus 100 according to the first embodiment in that it further includes a matching unit (not shown) for matching the payment information.

By including the payment information acquisition unit, the information processing apparatus 100 can generate more accurate preference information. By including the temporary ID assigning unit, the information processing apparatus 100 can temporarily distinguish face information from other face information. Further, by further including a collation unit, it is possible to collate the temporary ID and the user's payment information.

In Embodiment 1, a mode in which a user purchases a product using an EC service has been described, but in this modified example, a mode in which a user purchases a product at a physical store will be described.

In the first embodiment, the information processing apparatus 100 first acquires user's personal information. For example, in a shopping site, this corresponds to a user entering an ID and a password to log in to the shopping site.

In this modification, for example, the information processing device acquires the user information after acquiring the payment information from the payment information acquiring unit or at the same time as the payment information. As an example, an imaging unit is installed on a product display shelf or the like in a store so that the user's line of sight, blink, pupil state, and the like when the user is selecting products can be detected. Until the user information is acquired, for example, the temporary ID assigning unit assigns a temporary ID to the user, and stores information such as product information visually recognized by the user in a cache memory or the like as temporary ID information. may Payment information and user information are acquired for the first time when a user makes a payment at a cash register using a medium with which user information is linked, such as a credit card. For example, an imaging unit is also installed at the cash register, and after synchronizing the image acquired from the imaging unit of the display shelf with the image acquired from the imaging unit of the cash register, it is possible to determine which temporary ID user made the payment. The matching unit may match the temporary ID information and the user's payment information, or the temporary ID information and the user information.

It should be noted that if the user of the temporary ID makes a payment by a method that cannot be compared with the user information, such as when the payment is made in cash, the temporary ID information may be discarded.

<Embodiment 2>
The information processing device according to this embodiment is a learning support device. As a form of the learning support device, for example, there is a device that implements E-learning. In this embodiment, the information processing device generates an estimated degree of comprehension of the learner who answers the multiple choice questions.

FIG. 13 is an example of question sentences and answer options displayed by the information processing apparatus according to the present embodiment. As shown in FIG. 13, the learner reads the displayed question sentence, reads the answer options, and selects the option that he/she thinks is the correct answer. In such multiple-choice questions, even if the learner finally selects the correct option, the learner does not necessarily understand the question and answer correctly. As a result of getting lost between multiple options, you may accidentally get the correct answer, or as a result of choosing an option without reading the question text, you may accidentally get the correct answer. The information processing apparatus according to the present embodiment outputs the estimated comprehension level of the learner in consideration of the process of the learner reaching the answer. By outputting the estimated degree of comprehension in this way, the learner himself/herself or a third party who instructs the learner can accurately know the degree of understanding of the learner.

As in the first embodiment, the information processing apparatus according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, a preference information generation unit 6, and a program execution unit 30 .

The user information acquisition unit 1 acquires information on a pre-registered learner as a user.

The face information acquisition unit 2 acquires face information of the user. Acquisition of the user's face information performed by the face information acquisition unit 2 may use the same processing as that of the face information acquisition unit 2 of the first embodiment.

The state detection unit 4 detects the state of the user from the user's face information acquired by the face information acquisition unit 2 . As for the detection method, the same determination method as that of the state detection unit 4 of the first embodiment may be used.

The visible object determination unit 5 refers to the result detected by the state detection unit 4 and determines the object visually recognized by the user. The visible object determination unit 5 may further refer to the layout information of the page that the user is viewing to determine the object that the user is viewing.

Also, the visible object determination unit 5 may refer to the result detected by the state detection unit 4 and further determine at least one of the user's degree of concentration and the user's emotion. The determination method performed by the visible object determination unit 5 may be the same method as the determination method of the visible object determination unit 5 of the first embodiment. In this embodiment, the object visually recognized by the user is the question text and answer options visually recognized by the user. In this embodiment, the user's emotion is the learner's emotion when solving the problem, such as being confident or embarrassed.

In this embodiment, the preference information generated by the preference information generation unit 6 is the estimated degree of comprehension of the user who answered a certain question. Here, the estimated degree of comprehension is a degree for estimating the degree of comprehension of the learner (user) after considering only the process by which the learner (user) reached the answer. The preference information generation unit 6 refers to the user information acquired from the user information acquisition unit 1 and the determination result acquired from the visible object determination unit 5 to generate an estimated understanding level of the learner. As an example, the visual recognition object determination unit 5 determines that the learner is looking at an object many times, that is, the answer option that the learner has read many times. It is determined that the candidate is a questionable choice, and an estimated degree of comprehension is generated. In the present embodiment, the preference information generator 6 generates the learner's estimated degree of comprehension from the user information and the determination result without considering which option the user actually selected.

Thereby, a learner's estimated comprehension degree can be suitably generated.

FIG. 14 shows an example of information generated by the preference information generation unit 6. As shown in FIG. FIG. 14 shows an example of information generated by the preference information generation unit 6. As shown in FIG. In FIG. 14, "ID" is information for identifying the learner; "Option" is an option determined that the learner did not visually recognize so much. The "estimated degree of comprehension" is the result of comprehensively judging the "choices that were confused" and the "choices that were not hesitated" and showing the comprehension of the learner in 5 stages. In this example, there are four answer options a to d, and the correct answer option is a.

The estimated comprehension levels of the learners A to D will be explained below.

Since the learner A visually recognized the options b and c in addition to the correct option a, it can be said that the learner A hesitated among the options a, b, and c. Conversely, since learner A did not visually recognize option d that much, it can be said that there is a high possibility that learner A read option d and excluded it from the correct candidates. From this, the preference information generation unit 6 determines that learner A has enough understanding to exclude option d from the correct candidates, and generates an estimated understanding of "3" as an intermediate level of understanding.

It can be said that learner B was at a loss between options a and b because he visually recognized option a and option b, which are the correct answers, and did not visually recognize options c and d so much. Since either option a or b is narrowed down to be the correct answer, the preference information generation unit 6 sets the estimated understanding level of learner B to "4", which is higher than that of learner A. FIG.

Learner C read the question sentences and options, and immediately answered without hesitation. From this, the preference information generation unit 6 determines that the degree of understanding of the learner C is high, and generates an estimated degree of understanding of "5".

Since learner D visually recognizes options b, c, and d other than option a, it can be said that option a, which is the correct answer, was not included as a candidate for the correct answer from the beginning, and was at a loss between options b, c, and d. . From this, the preference information generation unit 6 determines that the degree of understanding of the learner D is not so high, and generates an estimated degree of understanding of "2".

It can be said that the learner E visually recognized all the options in the same way and hesitated over all the options. Based on this, the preference information generation unit 6 determines that the degree of understanding of the learner E is not so high, and generates an estimated degree of understanding "1".

(Program execution unit 2-1)
The program execution unit 30 may output display data including an object that visualizes the estimated comprehension level generated by the preference information generation unit 6 to an external display device. An example of display data is a graph.

FIG. 15 is a bar graph showing the estimated degree of comprehension of a learner who answered five question sentences. The visualization method is not limited, and a radar chart or the like may be used. Moreover, although FIG. 15 shows the estimated comprehension level of one learner, the average estimated comprehension level of a plurality of learners who have solved the same problem may be displayed together.

This makes it possible to visualize the estimated comprehension level of the learner.

In this embodiment, the form of the preference information generated by the preference information generation unit 6 is not particularly limited. As shown in FIG. 14, it may be information indicating the estimated understanding of multiple learners for each question, or as shown in FIG. 15, showing the estimated understanding of multiple questions answered by each learner It may be information.

For example, the information processing apparatus according to the present embodiment may further determine the following items to generate an estimated comprehension level.

If the learner answered in less than the minimum threshold time required to read the entire question and answer option sentences, the learner answered without reading the question and answer option sentences in the first place. It can be said that the possibility is high and it is not an answer based on understanding. In this case, the preference information generation unit 6 may perform processing such as lowering the estimated comprehension level.

Further, after weighting the answer options, if the learner is unsure of which answer options to choose, the weight of the option may be reflected in the estimated comprehension level. For example, when an answer option that a learner who should have studied in advance cannot normally choose is selected as an answer candidate, the preference information generation unit 6 may perform processing such as lowering the estimated degree of comprehension.

(Program execution unit 2-2)
The program execution unit 30 proposes to the user proposal information based on the preference information acquired from the preference information generation unit 6 . In the present embodiment, the suggestion information is, for example, made the learner review a question with a low estimated degree of understanding among the plurality of questions answered by each learner, which is included in the preference information generated by the preference information generation unit 6. It is information for Specifically, the information for reviewing is a commentary or the like for making the learner understand a question with a low estimated degree of comprehension. More specifically, the information for reviewing is a message such as "It seems that your understanding of the past perfect tense of English grammar is low.", "Please review pages 10 to 13 of the textbook again." There may be.

As a method of proposing the suggestion information, for example, after the learner finishes solving a plurality of predetermined questions, the program execution unit 30 may display the suggestion information on the display unit on the screen for confirming whether the answers are correct or not. Further, the program execution unit 30 may transmit the generated proposal information to the learner using communication means such as e-mail, social network service (SNS), and instant message (IM).

In this way, the program execution unit 30 proposes the suggested information to the learner, so that the learner can receive information based on his/her estimated degree of comprehension.

The program execution unit 30 may determine whether or not the user has viewed the suggested information. The determination as to whether or not the user has viewed the suggested information is made by, for example, the program execution unit 30 sending information as to whether or not the learner has viewed a screen displaying information for reviewing questions proposed to the learner. may be determined by referring to Information as to whether or not the learner visually recognizes the object is acquired from the visually recognized object determination unit 5, for example. The visible object determination unit 5 may perform, for example, the processing described in the (visible object determination unit) of the first embodiment.

Also, the program execution unit 30 may perform evaluation based on the result of the determination described above. Here, the evaluation is, for example, an evaluation as to whether or not the information (suggestion information) proposed to the learner for reviewing the question was appropriate. As an example, the program execution unit 30 evaluates by referring to information such as whether the learner who visually recognized the information for reviewing the problem logged in to the site for solving the problem again or actually solved the problem. I do. As an example, the evaluation value may be the ratio of the number of learners who solved the problem again to the total number of learners who proposed information for reviewing the problem.

(Modification 2-1)
The information processing apparatus 100 may further include a selection information acquisition section (not shown). The selection information acquisition unit acquires user selection information. In this embodiment, the selection information is the number of the option selected by the learner.

The information processing apparatus according to this modification includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, a selection information acquisition unit, and a preference information generation unit 6. I have.

The functions of the user information acquisition unit 1, the face information acquisition unit 2, the state detection unit 4, and the visible object determination unit 5 are the same as the functions of the members described in the second embodiment, so description thereof will be omitted.

The preference information generation unit 6 refers to the user information acquired from the user information acquisition unit 1, the determination result acquired from the visible object determination unit 5, and the selection information acquired from the selection information acquisition unit, and estimates the learner's degree of understanding. to generate As an example, the visual recognition object determination unit 5 determines that the learner is looking at an object many times, that is, the answer option that the learner has read many times. It is determined that the candidate is a questionable choice, and an estimated degree of comprehension is generated.

FIG. 16 shows an example of information generated by the preference information generation unit 6. As shown in FIG. In FIG. 16, "ID" is information for identifying the learner, "Selection number" is the number of the option actually selected by the learner of each ID, and "Correctness" is the correctness of the answer. “Choices that were lost” are choices that the learner has visually recognized in the visible object determination unit 5. It is an option. The "estimated degree of comprehension" is the result of comprehensively judging "right or wrong," "questionable options," and "not-questionable options," and indicating the learner's level of understanding on a five-level scale. In this example, there are four answer options a to d, and the correct answer option is a.

In FIG. 16, the estimated comprehension levels of learners A, C, and D who answered the question correctly will be described. Learners A, C, and D all selected option a, which is the correct answer, but the processes leading to the selection of option a are different.

Since learner A visually recognized options b and c in addition to option a, it can be said that learner A hesitated among options a, b, and c. Conversely, since learner A did not visually recognize option d that much, it can be said that there is a high possibility that learner A read option d and excluded it from the correct candidates. From this, the preference information generation unit 6 determines that learner A has enough understanding to exclude option d from the correct candidates, and generates an estimated understanding of "3" as an intermediate level of understanding.

Learner C read the question and the options, visually recognized only option a, and selected option a as the correct answer. From this, the preference information generation unit 6 determines that the degree of understanding of the learner C is high, and generates an estimated degree of understanding of "5".

Since the learner D visually recognized the options b, c, and d in addition to the option a, it can be said that the learner A hesitated over all the options. From this, learner D finally selected the correct option, but was at a loss with all the options. and generate an estimated comprehension level of "2".

Also, the estimated comprehension levels of learners B and E who answered the question incorrectly are determined in the same manner. For example, although learner B gave an incorrect answer, he hesitated between options a and b, and did not hesitate about other options. determines and produces an estimated comprehension of "4". In addition, since learner E was at a loss with all the options and ended up with an incorrect answer, the preference information generation unit 6 determines that the degree of understanding is low, and generates an estimated degree of understanding of "1".

Also in this modification, as described in <Embodiment 2>, the program execution unit 30 proposes proposal information based on the estimated understanding level, and performs evaluation based on the estimated understanding level and the proposal information. good.

The evaluation here is, as in <Embodiment 2>, whether or not the information (proposed information) proposed to the learner for reviewing the question was appropriate. In this modified example, when the learner who has visually confirmed the information for reviewing the problem solves the problem again, information such as whether the learner's performance has improved or not is further considered for evaluation. may

<Embodiment 3>
The information processing device 100 of this embodiment shares preference information with other terminal devices. FIG. 17 is a block diagram showing components of the information processing apparatus 100. As shown in FIG. As shown in FIG. 17, the information processing apparatus 100 according to the present embodiment includes a user information acquisition unit 1, a face information acquisition unit 2, a state detection unit 4, a visible object determination unit 5, a preference information generation unit 6, a program execution unit 30 , an interest sharing request transmitting section 20 , an interest sharing request receiving section 21 , an interest sharing request approval section 22 , and an interest sharing section 23 . The functions of members other than the preference sharing request transmission unit 20, the preference sharing request reception unit 21, the preference sharing request approval unit 22, and the preference sharing unit 23 are the same as the functions of the members described in the first embodiment. omitted.

FIG. 18 is a diagram showing an outline of an information processing system according to this embodiment. As shown in FIG. 18, the information processing system according to the present embodiment includes a server 300 and a plurality of terminal devices (10, 10a) as an example of a plurality of information processing apparatuses 100. As shown in FIG.

The preference sharing request transmission unit 20 transmits information for requesting sharing of preference information to other terminals via the server 300 .

The preference sharing request reception unit 21 receives information indicating that the preference information is requested to be shared from the preference sharing request transmission unit of another terminal via the server 300 . The taste sharing request receiving unit 21 notifies the received information to the taste sharing request approval unit.

Based on the information received from the preference sharing request receiving section 21, the preference sharing request approval section 22 causes the display section to display a notice asking the user for permission to transmit the preference information to another terminal. Based on this display, the user performs an operation input indicating whether or not to approve, and the preference sharing request approval section 22 transmits the input information to the preference sharing section 23 .

If the information received from the preference sharing request approval unit 22 is information indicating authorization, the preference sharing unit 23 acquires the preference information from the preference information generation unit 6, and transmits the preference information to other users via the server 300. Send to the terminal device. Further, the preference sharing unit 23 receives preference information from another terminal via the server 300, combines the preference information of its own terminal device and the preference information of the other terminal, and generates the preference information of both. At least part of the preference information of both parties is displayed on the display unit 12 . Specifically, the preference information of both terminals may be displayed on the terminal device, or only the preference information from the other terminal may be displayed on the own terminal device. Also, the preference information of both terminals and only the preference information from the other terminal may be switched and displayed as appropriate by the user's operation or the like. With such a configuration, the user can know the preference information of the other party who is the user of the other terminal. In addition, it is possible to know the taste information of both parties considering the taste information of the user and the taste information of the other party.

The server 300 manages the information for requesting sharing of the preference information in association with the terminal ID or user ID of the terminal device of the transmission source and the terminal ID or user ID of the terminal device of the transmission destination. . In addition, the server 300 manages the “taste information” in association with the terminal ID or user ID of the terminal device from which the taste information is acquired and the terminal ID or user ID of the terminal device with which the taste information can be shared. there is

As an example, a case of sharing taste information of restaurants will be described. When user A and user B select a restaurant to use together, user A and user B share preference information about each other's restaurant so that their preferences are the closest or the same. You can choose a restaurant.

For example, the taste sharing request transmission unit of the user A's terminal device transmits information for requesting user B's terminal device to share the taste information of restaurants of user B. FIG. The taste sharing request receiving unit 21 of the user B's terminal device notifies the taste sharing request approval unit 22 of the information received from the user A's taste sharing request transmitting unit, and displays the taste information on the display unit of the user B's terminal device. Let user B be notified asking for permission to transmit to other terminals. When the user B makes an input for approval, the preference sharing request approval unit 22 transmits information indicating the approval to the preference sharing unit 23 . The preference sharing unit 23 acquires the preference information from the preference information generation unit 6 and transmits the preference information to the user A's terminal device. Further, the preference information generation unit 6 acquires the preference information from the terminal device of the user A, generates the preference information of the two users based on the preference information of the user A and the user B, and displays the generated result on the display unit 12. Let

(Program execution unit 3)
As described in (program execution unit 1-1) of the first embodiment, the program execution unit 30 generates an avatar image corresponding to the user, the user state detected by the state detection unit 4, and the visible object determination unit 5 generates an avatar image corresponding to at least one of the object visually recognized by the user determined by , the user's emotion, and the preference information acquired from the preference information generation unit 6 .

In this embodiment, the program execution unit 30 may display the generated avatar image on the display unit 12 of the terminal device of the user so that the user can see the image, or the preference sharing unit 23 You may transmit to another terminal device via. That is, a plurality of users may share the avatar image together with the preference information.

In this way, the program execution unit 30 generates an avatar image, making it possible to visualize the object that the user is visually recognizing, the user's emotion, and preference information. In addition, by sharing avatar images among a plurality of users, it is possible to increase a sense of familiarity between users while looking at the display section of the terminal device.

FIG. 19 is an example of displaying user A's and user B's preference information. For example, a store included in both user A's preference information and user B's preference information is displayed at a higher rank as preference information common to both. In FIG. 19, for example, stores Y, P, X, and C displayed at the top of the browser screen 16 are stores that are common favorites of user A and user B. In FIG. Further, when the browser screen 16 is displayed on the display unit 12 of the user A's terminal device, only the user B's favorite stores may be displayed on the display unit 12 of the user A's terminal device.

The program execution unit 30 of this embodiment may change the layout of the display unit 12 as appropriate. The processing of the program execution unit 30 is the same as the processing of the program execution unit 30 described in the first embodiment.

The number of terminals that share the preference information is not particularly limited, and the information may be shared among three or more terminal devices.

In this embodiment, the terminal device includes the preference information generation unit 6 and the preference sharing unit 23, and the preference information is shared among a plurality of terminal devices. However, the present invention is not limited to this form. In the example shown in FIG. 18, a server 300 includes a preference information generation unit 6 and a preference sharing unit 23, and multiple users share their preference information via the server to generate preference information for a plurality of users. may be in the form

A configuration in which the server includes the preference information generation unit 6 will be described in a fourth embodiment.

<Embodiment 4>
[Main Configurations of Information Processing Device 400 and Server 200]
An information processing apparatus according to this embodiment will be described with reference to FIG. FIG. 20 is a block diagram showing components of a terminal device 10 including an information processing device 400 according to this embodiment and a server 200 which is an information processing device according to this embodiment.

The terminal device 10 acquires the user information and face information of the user, transmits the acquired information to the server 200, and the server 200 refers to the received information to generate preference information. The server 200 transmits the generated preference information to the terminal device 10, and the terminal device 10 acquires the preference information.

The information processing apparatus 400 according to the present embodiment includes a user information acquisition unit 1 that acquires user information of a specific registered user, a face information acquisition unit 2 that acquires face information of the user, user information of the user, and A preference information acquisition unit 24 for acquiring preference information obtained by referring to face information, and a program execution unit 30 are provided. Moreover, the terminal device 10 including the information processing device further includes a display unit.

Thereby, the information processing apparatus 400 according to the present embodiment can acquire the user's preference information.

The user information acquisition unit 1 and the face information acquisition unit 2 acquire the user information and the user's face information, respectively, as in the first embodiment.

User information includes user attribute information, and specific user attribute information is the same as the attribute information described in the first embodiment.

Further, the user information includes user identification information for identifying the user from other users, and specific user identification information is the same as the identification information described in the first embodiment.

This makes it possible to acquire user attributes. In addition, since the user identification information is included in the user information, the target user can be identified from other users, and the user's preference information and the user information can be linked.

This embodiment differs from the first embodiment in that the information acquired by the user information acquisition unit 1 and the face information acquisition unit 2 is transmitted to the server 200 as shown in FIG.

Further, the preference information acquisition unit 24 acquires the preference information from the server 200 and causes the display unit 12 to display the acquired result.

The information processing apparatus 400 may further include a program execution unit (not shown) that executes a program based on preference information.

Thereby, the information processing apparatus 400 can execute a program based on the user's preference information.

(Program execution unit 4-1)
The program execution unit 30 generates an avatar image corresponding to the user and corresponding to at least one of the user's state, the object the user is viewing, the user's emotion, and preference information. The details of the avatar image generation performed by the program execution unit are as described in the first embodiment.

In this way, the program execution unit 30 generates an avatar image, making it possible to visualize the object that the user is visually recognizing, the user's emotion, and preference information.

The program execution unit 30 may propose proposal information based on the preference information to the user, and perform evaluation based on the preference information and the proposal information. The details of the proposal and evaluation of the proposal information performed by the program execution unit are as described in the first embodiment.

In this way, the program execution unit 30 proposes the proposal information to the user, so that the user can enjoy information based on his or her taste. In addition, evaluation by the program execution unit 30 makes it possible to evaluate whether or not the proposal information is appropriate.

The program execution unit may change the display information to be displayed on the external display device based on the preference information. The details of the change of display information performed by the program execution unit are as described in the first embodiment.

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

[Main Configuration of Server 200]
The server 200 will be described based on FIG. The server 200 according to the present embodiment includes a user information acquisition unit 1a that acquires user information of a specific registered user, a face information acquisition unit 2a that acquires face information of the user, user information of the user, A preference information generation unit 6a for generating preference information about the user's preference by referring to the user's face information and a program execution unit 30a for executing a program based on the preference information.

Thereby, the server 200 can suitably generate the user's preference information.

As shown in FIG. 20, the server 200 according to this embodiment further includes a transmission unit 19 that transmits preference information.

A user information acquisition unit 1a and a face information acquisition unit 2a provided in the server 200 receive user information and face information from the user information acquisition unit 1 and the face information acquisition unit 2 in the terminal device 10, respectively, and use the received information as a preference. It is transmitted to the information generator 6a.

The preference information generation unit 6a generates user preference information by referring to the received user information and face information. The specific operation of the preference information generation unit 6a is the same as that of the preference information generation unit 6 described in the first embodiment, but as shown in FIG. , a state detection unit 4a, and a visible object determination unit 5a. Here, the functions of the state detection unit 4a and the visible object determination unit 5a are the same as the functions of the state detection unit 4 and the visible object determination unit 5 described in the first embodiment.

By including the state detection unit 4a and the visible object determination unit 5a in this way, the server 200 can detect the state of the user and suitably determine the object visually recognized by the user.

The information processing apparatus 400 that performs transmission/reception with the server 200 refers to a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit 2 and the result detected by the state detection unit. A visible object determination unit for determining a visually recognized object may be provided.

In this way, by including the state detection unit and the visible object determination unit, the information processing apparatus 400 can detect the user's state and suitably determine the object visually recognized by the user.

Similar to the visible object determination unit according to the first embodiment, the visible object determination unit further refers to the layout information of the page viewed by the user to determine the object viewed by the user.

In this way, the visible object determining unit further refers to the page layout information, so that it is possible to accurately determine which object on the page the user is visually recognizing.

Also, the visible object determination unit may refer to the result detected by the state detection unit, and further determine at least one of the user's degree of concentration and the user's emotion. The function of the visible object determination unit is the same as that of the visible object determination unit 5 according to the first embodiment.

This makes it possible to determine to what extent the user is interested in the object and with what kind of emotion the user is visually recognizing the object.

The information processing apparatus 100 may further include an image acquisition unit (not shown) that acquires a captured image including the user's field of view from an imaging device worn by the user. Further, the visible object determination unit may determine the object visually recognized by the user by referring to the captured image acquired by the image acquisition unit and the result detected by the state detection unit.

The functions of the image acquisition unit and the visible object determination unit are the same as the functions of the image acquisition unit 15 and the visible object determination unit 5 described in the first embodiment.

Thereby, the object visually recognized by the user can be suitably determined.

The state of the user detected by the state detection unit is the same as the state detection unit 4 according to the first embodiment, and includes the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, At least one of nose movement, lip movement and jaw movement.

As a result, the degree of concentration of the user can be suitably determined.

As in the first embodiment, the preference information is information about objects in which the user has a strong interest and objects in which the user has a weak interest.

As a result, the preference information generation unit 6a provided in the server 200 can know the information of the object in which each user is strongly interested and the object in which the user is weakly interested, and the preference information generation unit generates weighted preference information. be able to. In addition, the user can acquire the preference information based on his/her own preference by the preference information acquisition unit 24 provided in the information processing apparatus 400 acquiring the preference information.

The form in which the information processing apparatus is a server and the preference information generation unit provided in the server generates the preference information is not limited to this embodiment, and can be applied to the first to third embodiments described above.

By providing the server with the preference information generation unit in this way, it is possible to generate the user's preference information on the server even if the terminal device used by the user is not always the same. As long as the user information and face information of the user can be obtained, the preference information of the user can be stably generated regardless of whether the user uses a terminal device such as a personal computer, a tablet, or a smart phone.

In addition, it is particularly useful for the server to include the preference information generation unit in the form of sharing the preference information of a plurality of users described in the third embodiment. When sharing preference information of a plurality of users, for example, the server may further include a preference sharing unit. With such a configuration, it is possible to suitably generate the preference information of a plurality of users on the server.

(Program execution unit 4-2)
The program execution unit 30a generates an avatar image corresponding to the user, which corresponds to at least one of the user's state, the object the user is viewing, the user's emotion, and preference information. The processing performed by the program execution unit 30a is the same as the processing described in (program execution unit 1-1) of the first embodiment.

This makes it possible to visualize the object visually recognized by the user, the user's emotion, and preference information.

Note that the avatar image generated by the program execution unit 30 a of this embodiment is transmitted to the preference information acquisition unit 24 of the terminal device 10 via the transmission unit 19 . Although FIG. 20 shows a form in which transmission is performed only to the terminal device 10, the type and number of transmission destination devices are not particularly limited, and transmission may be performed to a plurality of devices.

In this way, by the server transmitting the avatar image generated by the program execution unit 30a to a plurality of devices, a plurality of users can view the avatar image.

Also, in FIG. 20, the face information and user information are received only from the terminal device 10, but the type and number of devices that transmit the face information and user information are not particularly limited. Facial information and user information may be received from multiple devices. In this case, the preference information generation unit 6a of the server 200 may generate the preference information of each user by referring to the face information and user information of a plurality of users. Further, the preference information generation unit 6a may generate preference information by integrating the preference information of a plurality of users by referring to the face information and user information of a plurality of users. The program execution unit 30a may refer to the preference information of each user generated by the preference information generation unit 6a to generate an avatar image corresponding to each user. Also, the program execution unit 30a may be configured to be able to transmit an avatar image corresponding to a certain user to a terminal device associated with another user. As an example, the program execution unit 30a generates an image that displays avatar images corresponding to a plurality of users in one virtual space, and transmits the image to each terminal device associated with the plurality of users. It is good also as possible composition. More specifically, the program execution unit 30a arranges an avatar image corresponding to each user in a virtual space modeled after an actual store, and creates an image that makes it look like each user is shopping at the actual store. may be generated.

In this way, the server receives face information and user information of a plurality of users from a plurality of devices, so that the preference information generating section 6a can generate preference information of a plurality of users. Also, the program execution unit 30a can generate an avatar image by referring to preference information of a plurality of users.

The program execution unit 30a may perform evaluation based on the preference information generated by the preference information generation unit 6a and the proposal information proposed by the program execution unit 30a. The details of the proposal and evaluation of the proposal information performed by the program execution unit 30a are as described in the first embodiment.

In this way, the program execution unit 30a proposes the proposal information to the user, so that the user can enjoy information based on his or her taste. In addition, evaluation by the program execution unit 30a makes it possible to evaluate whether the proposal information is appropriate.

The program execution unit 30a may change the display information to be displayed on the external display device based on the preference information. The details of the change of display information performed by the program execution unit 30 are as described in the first embodiment.

Thereby, it is possible to provide each user with a layout or the like based on each user's preference information.

[Example of realization by software]
The control block of the information processing apparatus 100 may be implemented by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be implemented by software.

In the latter case, the information processing apparatus 100 is provided with a computer that executes instructions of a program, which is software that implements each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium storing the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

The information processing device according to each aspect of the present invention may be realized by a computer. A control program for an information processing apparatus realized by a computer and a computer-readable recording medium recording it are also included in the scope of the present invention.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

1, 1a user information acquisition unit 2, 2a face information acquisition unit 4, 4a state detection unit 5, 5a visible object determination unit 6, 6a preference information generation unit 15 image acquisition unit 30, 30a program execution unit 24 preference information acquisition unit 100 , 400 information processing device

Claims (37)

a user information acquisition unit that acquires user information of a specific registered user;
a face information acquisition unit that acquires face information of the user;
a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit;
A visible object determination unit that refers to the result detected by the state detection unit and determines an object that the user is viewing;
a comprehension level information generation unit that generates an estimated comprehension level of the user regarding learning by referring to the user's state by the state detection unit and the determination result by the visible object determination unit;
a program execution unit that executes a program based on the estimated degree of comprehension;
and
The above program execution part
Proposing suggested information based on the estimated degree of understanding to the user,
An information processing apparatus , wherein an evaluation is performed based on the estimated degree of understanding and the proposal information . 2. The information processing apparatus according to claim 1, wherein said estimated degree of comprehension is said user's estimated degree of comprehension of a selection question. 3. The information processing apparatus according to claim 2, wherein said estimated degree of comprehension includes an answer number selected by said user. 4. The information processing apparatus according to claim 3, wherein said comprehension level information generation unit generates said estimated comprehension level by referring to whether the number of said answer finally selected by said user is correct or not. The visible object determination unit classifies the object into either an undecided option or an unambiguous option according to the degree of recognition by the user of the object,
Claims 1-, wherein the comprehension level information generating unit generates the estimated comprehension level by further referring to at least one selected from the group consisting of the user's answer being correct or not, the choice that the user hesitated about, and the option that he did not hesitate about. 5. The information processing apparatus according to any one of 4. The visible object determination unit
6. The information processing apparatus according to any one of claims 1 to 5, wherein the object viewed by the user is determined by further referring to layout information of the page viewed by the user. . The visible object determination unit
7. The method according to any one of claims 1 to 6, wherein at least one of the degree of concentration of the user and the emotion of the user is further determined with reference to the result detected by the state detection unit. Information processing equipment. Further comprising an image acquisition unit that acquires a captured image including the user's field of view from the imaging device worn by the user,
The visible object determination unit
a captured image acquired by the image acquiring unit;
a result detected by the state detection unit;
8. The information processing apparatus according to any one of claims 1 to 7, wherein the object viewed by the user is determined by referring to. The above program execution part
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the user's state, an object viewed by the user, the user's emotion, and the estimated degree of understanding. 5. The information processing apparatus according to any one of claims 1 to 4. The above program execution part
10. The information processing apparatus according to any one of claims 1 to 9 , wherein the estimated comprehension level is output as display data to an external display device. The above program execution part
11. The information processing apparatus according to any one of claims 1 to 10 , wherein display data including an object that visualizes the estimated degree of comprehension is output to an external display device. The above program execution part
12. The information processing apparatus according to any one of claims 1 to 11 , wherein display information to be displayed on an external display device is changed based on said estimated degree of comprehension. The information processing apparatus according to any one of claims 1 to 12 , wherein said user information includes attribute information of said user. The information processing apparatus according to any one of claims 1 to 13 , wherein the user information includes user identification information for identifying the user from other users. The state of the user detected by the state detection unit includes the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. 15. The information processing apparatus according to any one of claims 1 to 14 , wherein at least one of An information processing program for causing a computer to function as the information processing apparatus according to claim 1,
An information processing program for causing a computer to function as the user information acquisition unit, the face information acquisition unit, the state detection unit, the visible object determination unit, the comprehension level information generation unit, and the program execution unit. a user information acquisition step of acquiring user information of a specific registered user;
a face information obtaining step of obtaining face information of the user;
a state detection step of detecting the state of the user from the face information acquired in the face information acquisition step;
A visible object determination step for determining an object viewed by the user by referring to the result detected in the state detection step;
a comprehension level information generating step for generating an estimated comprehension level of the user regarding learning by referring to the user information of the user and the determination result in the visible object determining step;
a program execution step of executing a program based on the estimated degree of comprehension;
contains
The above program execution step is
Proposing suggested information based on the estimated degree of understanding to the user,
An information processing method , wherein an evaluation is performed based on the estimated degree of understanding and the proposed information . a user information acquisition unit that acquires user information of a specific registered user;
a face information acquisition unit that acquires face information of the user;
an understanding level information acquisition unit that acquires an estimated understanding level of the user regarding learning, which is obtained by referring to the user information of the user and the face information of the user;
a program execution unit that executes a program based on the estimated degree of comprehension;
and
The above program execution part
Proposing suggested information based on the estimated degree of understanding to the user,
An information processing apparatus , wherein an evaluation is performed based on the estimated degree of understanding and the proposal information . 19. The information processing apparatus according to claim 18 , wherein said estimated degree of understanding is said user's estimated degree of understanding of a selection question. 20. The information processing apparatus according to claim 19 , wherein said estimated degree of comprehension includes an answer number selected by said user. 21. The information processing apparatus according to claim 20 , wherein said comprehension level information acquisition unit acquires said estimated comprehension level by referring to whether the number of said answer finally selected by said user is correct or not. The above program execution part
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the user's state, an object viewed by the user, the user's emotion, and the estimated degree of understanding. The information processing apparatus according to any one of claims 18-21 . The above program execution part
23. The information processing apparatus according to any one of claims 18 to 22 , wherein display information to be displayed on an external display device is changed based on said estimated degree of comprehension. a state detection unit that detects the state of the user from the face information acquired by the face information acquisition unit;
A visible object determination unit that refers to the result detected by the state detection unit and determines an object that the user is viewing;
24. The information processing apparatus according to any one of claims 18 to 23 , further comprising: The visible object determination unit classifies the object into either an undecided option or an unambiguous option according to the degree of recognition by the user of the object,
25. The comprehension level information acquiring unit acquires the estimated comprehension level by further referring to at least one selected from the group consisting of whether the user's answer is right or wrong, the option that the user hesitates about, and the option that the user does not hesitate about. The information processing device described. The visible object determination unit
26. The information processing apparatus according to claim 24 , wherein the object viewed by the user is determined by further referring to layout information of a page viewed by the user. The visible object determination unit
27. The method according to any one of claims 24 to 26 , wherein at least one of the degree of concentration of the user and the emotion of the user is further determined with reference to the result detected by the state detection unit. Information processing equipment. Further comprising an image acquisition unit that acquires a captured image including the user's field of view from the imaging device worn by the user,
The visible object determination unit
a captured image acquired by the image acquiring unit;
a result detected by the state detection unit;
28. The information processing apparatus according to any one of claims 24 to 27 , wherein the object viewed by the user is determined by referring to. The state of the user detected by the state detection unit includes the user's line of sight, pupil state, number of blinks, eyebrow movement, eyelid movement, cheek movement, nose movement, lip movement, and jaw movement. 29. The information processing apparatus according to any one of claims 24 to 28 , wherein at least one of The information processing apparatus according to any one of claims 24 to 29 , wherein said user information includes attribute information of said user. The information processing apparatus according to any one of claims 24 to 30 , wherein said user information includes user identification information for identifying said user from other users. a user information acquisition unit that acquires user information of a specific registered user;
a face information acquisition unit that acquires face information of the user;
a comprehension level information generation unit that generates an estimated comprehension level of the user regarding learning by referring to the user information of the user and the face information of the user;
a program execution unit that executes a program based on the estimated degree of comprehension;
and
The above program execution part
Proposing suggested information based on the estimated degree of understanding to the user,
An information processing apparatus , wherein an evaluation is performed based on the estimated degree of understanding and the proposal information . 33. The information processing apparatus according to claim 32 , wherein said estimated degree of understanding is said user's estimated degree of understanding of a selection question. 34. The information processing apparatus according to claim 33 , wherein said estimated degree of comprehension includes an answer number selected by said user. 35. The information processing apparatus according to claim 34 , wherein said comprehension level information generation unit generates said estimated comprehension level by referring to the correctness of said answer number finally selected by said user. The above program execution part
Generating an avatar image corresponding to the user, the avatar image corresponding to at least one of the user's state, an object viewed by the user, the user's emotion, and the estimated degree of understanding. 36. The information processing apparatus according to any one of claims 32-35 . The above program execution part
37. The information processing apparatus according to any one of claims 32 to 36 , wherein display information to be displayed on an external display device is changed based on said estimated degree of comprehension.

Images