JP2016224142A - Evaluation method, program and evaluation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an evaluation method capable of precisely comprehending the concentration level of a user to a display object.SOLUTION: The evaluation method in an embodiment includes the steps of: (A) identifying a first point of fixation of a user in a first period of time when at least a first object is displayed in a screen; (B) after the first period of time, identifying a second point of fixation of the user in a second period when at least a second object is displayed in the screen; and (C) calculating the degree of approximation between a direction of a first vector from a reference point in the first object to a reference point in the second object and a direction of a second vector from a first point of fixation to a second point of fixation.SELECTED DRAWING: Figure 11A

Description

  The present invention relates to a technique for evaluating a user's degree of concentration on display.

  One patent document describes that the degree of concentration of a student in a class is estimated based on the degree of correlation between the gazes of the teacher and the student.

  In this way, when trying to estimate the degree of concentration by capturing the learner's line of sight, if the line of sight cannot be correctly identified, the concentration degree estimate itself is unreliable.

JP 2001-228785 A International Publication No. 2007/086222

  In one aspect, an object of the present invention is to more accurately grasp a user's degree of concentration with respect to a display object.

  An evaluation method according to an aspect includes: (A) identifying a first gaze point of a user in a first period for displaying at least a first object in a screen; and (B) after a first period, at least a second object The second gaze point of the user in the second period to be displayed on the screen is specified, and (C) the direction of the first vector from the reference point in the first object to the reference point in the second object, and the first gaze point And a process of calculating the degree of approximation with the direction of the second vector from the second gazing point.

  As one aspect, the degree of concentration of the user with respect to the display object can be grasped more accurately.

FIG. 1 is a diagram showing a state of viewing. FIG. 2 is a diagram illustrating an example of screen transition. FIG. 3 is a diagram illustrating an example of a gazing point. FIG. 4 is a diagram illustrating the first vector. FIG. 5 is a diagram illustrating an example of the first vector and the second vector. FIG. 6 is a diagram illustrating an example of the first vector and the second vector. FIG. 7 is a diagram illustrating an example of the first vector and the second vector. FIG. 8 is a diagram illustrating a module configuration example of the information processing apparatus. FIG. 9A is a diagram illustrating an example of a presentation processing flow. FIG. 9B is a diagram illustrating an example of a presentation processing flow. FIG. 9C is a diagram illustrating an example of a presentation processing flow. FIG. 9D is a diagram illustrating an example of a presentation processing flow. FIG. 9E is a diagram illustrating an example of a presentation processing flow. FIG. 10 is a diagram for explaining the first period to the fourth period. FIG. 11A is a diagram illustrating an example of an evaluation process flow. FIG. 11B is a diagram illustrating an example of an evaluation process flow. FIG. 12 is a diagram illustrating an example of a first specific processing flow. FIG. 13 is a diagram for explaining the maximum frequency. FIG. 14 is a diagram illustrating an example of screen transition according to the second embodiment. FIG. 15A is a diagram showing an example of a presentation processing flow according to Embodiment 2. FIG. 15B is a diagram showing an example of a presentation process flow according to Embodiment 2. FIG. 16 is a diagram illustrating an example of screen transition according to the third embodiment. FIG. 17A is a diagram illustrating an example of a presentation processing flow according to Embodiment 3. FIG. 17B is a diagram showing an example of a presentation processing flow according to Embodiment 3. FIG. 17C is a diagram illustrating an example of a presentation processing flow according to Embodiment 3. FIG. 17D is a diagram showing an example of a presentation process flow according to Embodiment 3. FIG. 18 is a diagram illustrating an example of presentation processing according to the fourth embodiment. FIG. 19 is a diagram illustrating an example of the presentation process according to the fourth embodiment. FIG. 20 is a diagram illustrating an example of the presentation process according to the fifth embodiment. FIG. 21 is a diagram illustrating an example of the presentation process according to the fifth embodiment. FIG. 22 is a functional block diagram of a computer.

[Embodiment 1]
FIG. 1 shows a state of viewing. In the system according to the present embodiment, a content screen is displayed, and a presentation that outputs content audio is also performed. Below, the example of a learning system is demonstrated. The display device 101 displays a content screen. The speaker 102 outputs content audio. A line-of-sight sensor device 103 is provided above the display device 101. The line-of-sight sensor device 103 reads the eye movement of the viewer seated in front of the display device 101 and detects the viewer's line of sight. The display device 101, the speaker 102, and the line-of-sight sensor device 103 are connected to an information processing device. Note that the line-of-sight sensor device 103 may detect the viewer's line of sight by reading the orientation of the face.

  The information processing apparatus performs presentation processing based on the content data. Furthermore, the information processing apparatus measures which position on the display surface of the display apparatus 101 the point of the line of sight detected by the line-of-sight sensor device 103 indicates during the presentation. Based on the measurement result, the information processing apparatus obtains a value that is an index of the degree of concentration by the viewer (hereinafter referred to as an index value of the degree of concentration).

  In this way, the information processing apparatus also performs a process of evaluating whether the viewer is concentrated on the content, and further controls the flow of output upon receiving feedback of the evaluation result in the presentation process.

  The point on the display pointed to by the viewer's line of sight corresponds to the intersection of the viewer's line of sight and the display surface of the display device 101. Therefore, hereinafter, a point on the display pointed to by the viewer's line of sight is simply referred to as an intersection.

  In general, calibration is performed so that the intersection detection using the visual line sensor device 103 becomes accurate. However, the calibration is effective on the premise that the positional relationship between the viewer's eyes and the display device 101 is the same as when the calibration is performed. That is, even if the display device 101 does not move, if the viewer changes his / her posture while viewing the content, the positional relationship between the viewer's eyes and the display device 101 becomes different from the initial one. And the point which is actually gazing may differ from the detected intersection.

  In this embodiment, even if the viewer's eyes move in this way and the intersection cannot be detected correctly, the degree of concentration on the content can be evaluated more correctly.

  The information processing apparatus in this example is a personal computer or server apparatus separated from the display apparatus 101, the speaker 102, and the line-of-sight sensor apparatus 103. However, the information processing apparatus may be a notebook personal computer, a tablet terminal, or a smartphone integrated with the display device 101, the speaker 102, and the line-of-sight sensor device 103. Furthermore, the display device 101 may be a projection display device such as a projector.

  Next, an example of content presented by the learning system in this example will be described. FIG. 2 shows an example of screen transition. The content in this example is a learning material and describes the theme S and the theme T. The theme S includes item A and item B. The theme T includes items C and D. In this content, the theme S is explained first, then the theme T is explained, and finally, the test is performed according to the degree of concentration of the viewer.

  In this example, there are two themes, but three or more themes may be handled. Furthermore, the number of items included in the theme is not limited to two, and may be three or more.

  The first row of FIG. 2 shows a page for explaining the theme S. A first window 201 relating to the item A is provided at the upper left of the page for explaining the theme S. In the first window 201, an explanation for item A is displayed. However, an image for understanding the item A may be displayed together with the explanatory text or instead of the explanatory text.

  In the state where the first window 201 is displayed, a sound for explaining the item A is output. The voice may be a reproduction of data obtained by recording a speaker's voice, or may be a synthesized voice that reads out text data. The content creator intends to deepen understanding of the item A by listening to the audio explaining the item A while the viewer looks at the first window 201.

  When the output of the sound describing the item A is finished, a second window 203 relating to the item B is newly provided at the lower right of the page for explaining the theme S, as shown in the second row of FIG. In the second window 203, it is assumed that an explanation for item B is displayed.

  In the state where the second window 203 is displayed, a sound for explaining the item B is output. The content creator intends to deepen understanding of the item B by listening to the audio explaining the item B while the viewer looks at the second window 203.

  In this way, it is assumed that the viewer's gaze point shifts from the first window 201 to the second window 203 at the timing when the description of the item A is switched to the description of the item B.

  Then, when the output of the sound for explaining the item B is finished, the page for explaining the theme S is switched to the page for explaining the theme T as shown in the third row of FIG. In the upper right of the page for explaining the theme T, a third window 205 relating to the item C is provided. It is assumed that an explanatory text about item C is displayed in the third window 205.

  In the state where the third window 205 is displayed, a sound for explaining the item C is output. The content creator intends to deepen understanding of the item C by listening to the audio explaining the item C while the viewer looks at the third window 205.

  When the output of the sound for explaining the item C is finished, a fourth window 207 for the item D is newly provided at the lower left of the page for explaining the theme T as shown in the fourth row of FIG. In the fourth window 207, an explanation about the item D is displayed.

  While the fourth window 207 is displayed, a sound for explaining the item D is output. The content creator intends to deepen understanding of the item D by listening to the audio explaining the item D while the viewer looks at the fourth window 207.

  In this way, it is assumed that the viewer's gaze point shifts from the third window 205 to the fourth window 207 at the timing when the description of the item C is switched to the description of the item D.

  If the movement of the gazing point is in line with the content creation plan, it is assumed that the viewer is concentrated on the content and is learning as intended. On the other hand, if the movement of the gazing point is not in line with the content creation plan, it is assumed that the viewer is not focused on the content and is not learning as intended. Therefore, in the present embodiment, as described above, it is determined whether the viewer is concentrated on the content based on the movement of the viewer's gazing point at the timing when the description is switched, and the test is performed based on the determination result. I do.

  If the information processing apparatus determines that the viewer is not concentrated on the theme S based on the movement of the viewer's gazing point at the timing of switching from the first stage to the second stage in FIG. As shown on the left side of, a test on the theme S is performed. On the other hand, when the information processing apparatus determines that the viewer is concentrated on the theme S, the test on the theme S is not performed.

  Similarly, when the information processing apparatus determines that the viewer is not concentrated on the theme T based on the movement of the viewer's gazing point at the timing of switching from the third stage to the fourth stage in FIG. As shown on the right side of the fifth row, a test on the theme T is performed. On the other hand, when the information processing apparatus determines that the viewer is concentrated on the theme T, the test on the theme T is not performed. This is the end of the description of screen transition.

  In the present embodiment, for example, regarding the switching from the first stage to the second stage in FIG. 2, the distance and direction on the layout that guides the line of sight are represented by the first vector, and the movement of the gazing point based on the actual measurement Is represented by a second vector. Then, when the direction of the first vector and the direction of the second vector are approximated, it is estimated that the viewer is concentrated on the content.

  Next, a method for determining the point of interest by the viewer in the information processing apparatus will be described. The line-of-sight sensor device 103 performs detection several tens of times per second, for example. In addition, the detection result varies more greatly than the viewer is aware of. In this embodiment, based on a plurality of detected intersections, a point that is considered to be consciously viewed by the viewer is specified as the point of sight.

  FIG. 3 shows an example of a plurality of measured intersections and gaze points derived from the intersections. The origin of the coordinate system in this example is the upper left corner of the display surface. The positive direction of the X coordinate is set in the right direction, and the positive direction of the Y coordinate is set in the downward direction.

  This example shows an example of the first gazing point 305 in the state shown in the first row of FIG. 2 and the second gazing point 307 in the state shown in the second row of FIG.

  It is assumed that the first gazing point 305 is specified based on the intersection 301a, the intersection 301b, the intersection 301c, and the intersection 301d detected near the upper left of the display surface. Actually, more intersections 301 are detected in a wide range. Here, for convenience of explanation, it is assumed that four intersections 301 are focused.

  A peripheral area 303 is set at each of the intersection 301a, the intersection 301b, the intersection 301c, and the intersection 301d. Specifically, a peripheral area 303a is set for the intersection point 301a, a peripheral area 303b is set for the intersection point 301b, a peripheral area 303c is set for the intersection point 301c, and further, for the intersection point 301d. A peripheral area 303d is set.

  A region indicated by a thick line indicates a place where the frequency included in the peripheral region 303a, the peripheral region 303b, the peripheral region 303c, and the peripheral region 303d is the highest. In this example, the frequency at which this area overlaps the surrounding area is 4. The center of gravity of this region is the first gazing point 305.

  On the other hand, the second gazing point 307 is specified based on the intersection point 301e, the intersection point 301f, the intersection point 301g, and the intersection point 301h detected in the vicinity of the lower right of the display surface. A peripheral region 303e is set for the intersection point 301e, a peripheral region 303f is set for the intersection point 301f, a peripheral region 303g is set for the intersection point 301g, and a peripheral region 303h is further set for the intersection point 301h. Is set.

  A region indicated by a thick line indicates a place where the frequency included in the peripheral region 303e, the peripheral region 303f, the peripheral region 303g, and the peripheral region 303h is the highest. In this example, the frequency at which this area overlaps the surrounding area is 4. The center of gravity of this area is the second gazing point 307.

  In response to these results, the second vector 309 is determined. The second vector 309 has a first gazing point 305 as a start point and a second gazing point 307 as an end point.

  Next, the first vector 405 will be described with reference to FIG. The first vector 405 has a reference point 401 of the first window 201 as a start point and a reference point 403 of the second window 203 as an end point. The reference point 401 of the first window 201 in this example is the center of gravity of the first window 201. Similarly, the reference point 403 of the second window 203 is the center of gravity in the second window 203.

  When the viewer is concentrated on the content in the same posture as when the calibration was performed, the second vector 309 almost coincides with the first vector 405. That is, the first gazing point 305 substantially coincides with the reference point 401 of the first window 201, and the second gazing point 307 substantially coincides with the reference point 403 of the second window 203.

  On the other hand, if the viewer is not concentrated on the content, the second vector 309 does not match the first vector 405 even if the posture is the same as when the calibration was performed.

  An example of the first vector 405 and the second vector 309a when the viewer is not concentrated on the content is shown in FIG. In the state shown in the first row of FIG. 2, it is assumed that the viewer is mainly looking inside the first window 201. Therefore, the first gazing point 305 a is within the first window 201. However, when the viewer moves to the state shown in the second row in FIG. 2, the second gazing point 307 a is not within the second window 203 because the viewer looks away. In such a case, the direction of the first vector 405 and the direction of the second vector 309a are different.

  Next, it is assumed that the viewer is in a forward posture, that is, the viewer's eyes are closer to the display surface of the display device 101 than at the time of calibration. FIG. 6 shows an example of the first vector 405 and the second vector 309b in this state. In this example, the viewer mainly looks at the inside of the first window 201 in the state shown in the first row of FIG. 2, and mainly looks at the inside of the second window 203 in the state shown in the second row of FIG. It shall have been.

  The first gazing point 305b is shifted upward from the reference point 401. The second gazing point 307b is shifted from the reference point 403 in the upper right direction. The second vector 309b is larger than the first vector 405. Therefore, in any of the comparison of the position of the first gazing point 305b and the reference point 401, the comparison of the position of the second gazing point 307b and the reference point 403, and the comparison of the second vector 309b and the first vector 405, it is an approximation. do not do.

  However, the direction of the second vector 309b and the direction of the first vector 405 are approximate. This embodiment is based on this characteristic.

  FIG. 7 shows an example of the first vector 405 and the second vector 309c when the viewer pulls backward, that is, when the viewer's eye position is farther from the display surface of the display device 101 than during calibration. Indicates. In this example, the viewer mainly looks at the inside of the first window 201 in the state shown in the first row of FIG. 2, and mainly looks at the inside of the second window 203 in the state shown in the second row of FIG. It shall have been.

  The first gazing point 305c is shifted downward from the reference point 401. The second gazing point 307c is shifted leftward from the reference point 403. The second vector 309c is smaller than the first vector 405. Therefore, in any of the comparison of the position of the first gazing point 305c and the reference point 401, the comparison of the position of the second gazing point 307c and the reference point 403, and the comparison of the second vector 309c and the first vector 405, it is an approximation. do not do.

  However, the direction of the second vector 309c and the direction of the first vector 405 are approximate. This embodiment is based on this characteristic.

  Below, operation | movement of the information processing apparatus 801 is demonstrated. FIG. 8 shows a module configuration example of the information processing apparatus 801. The information processing apparatus 801 includes a presentation unit 803 and an evaluation unit 805.

  The presentation unit 803 makes a presentation. Specifically, the presentation unit 803 displays pages and windows based on the content data. Furthermore, the presentation unit 803 outputs a description voice based on the content data.

  The presentation unit 803 includes a display processing unit 811, an audio output unit 813, a first notification unit 815, a first determination unit 817, a reception unit 819, a first reception unit 821, and a content storage unit 823. The display processing unit 811 performs processing for displaying pages and windows. The audio output unit 813 performs a process of outputting audio. The first notification unit 815 notifies the evaluation unit 805. The first determination unit 817 performs various determinations. The receiving unit 819 receives a determination result regarding the concentration of viewers from the evaluation unit 805. The 1st reception part 821 receives an instruction | indication of a viewer. The content storage unit 823 stores content data. The content data includes, for example, image data of each page, explanatory text or image data displayed in each window, size and position of each window, and the like. In this example, the position represents the coordinates of the upper left end point of the window. The content data also includes data for generating explanation audio.

  The evaluation unit 805 evaluates the degree of viewer concentration on the content. The evaluation unit 805 includes a control unit 831, a second reception unit 833, a specification unit 835, a calculation unit 837, a second determination unit 839, a second notification unit 841, an output unit 843, a layout storage unit 845, and a frequency storage unit 847. .

  The control unit 831 controls the line-of-sight sensor device 103 to detect the intersection coordinates. The second reception unit 833 receives a notification from the presentation unit 803. The specifying unit 835 performs first to fourth specifying processes to be described later. The calculation unit 837 performs various calculations. The second determination unit 839 performs various determinations. The second notification unit 841 notifies the presentation unit 803 of the determination result regarding the concentration of the viewer on the content. The output unit 843 outputs a concentration index value. The layout storage unit 845 stores the size and position of each window. The frequency storage unit 847 is used to count the frequency at which each pixel overlaps the peripheral region 303 in first to fourth specifying processes to be described later.

  The presentation unit 803, the evaluation unit 805, the display processing unit 811, the audio output unit 813, the first notification unit 815, the first determination unit 817, the reception unit 819, the first reception unit 821, the control unit 831, and the second reception unit described above. 833, the identification unit 835, the calculation unit 837, the second determination unit 839, the second notification unit 841, and the output unit 843 are a hardware resource (for example, FIG. 22) and a program that causes the processor to execute the processing described below. It is realized using.

  The content storage unit 823, the layout storage unit 845, and the frequency storage unit 847 described above are realized using hardware resources (for example, FIG. 22).

  Let's move on to the processing. 9A to 9E show examples of the presentation process flow. The display processing unit 811 displays the theme S page on the screen of the display device 101 (S901). The page in this example consists of a theme name and a background image.

  The display processing unit 811 displays the first window 201 at a predetermined display position on the theme S page (S903). At this time, the brightness of the first window 201 may be increased. Or you may make it display the animation of the avatar which points the 1st window 201 with a finger | toe.

  The audio output unit 813 starts outputting audio that describes the item A (S905). At this time, a voice message instructing to pay attention to the first window 201 may be output at the beginning. Then, the first notification unit 815 notifies the evaluation unit 805 of the start of the first period (S907). This notification includes the size and position of the first window 201.

  Here, a sampling period for obtaining a gazing point will be described with reference to FIG. The passage of time is shown in the right direction. Hereinafter, a sampling period for obtaining the first gazing point 305 for the first window 201 is referred to as a first period. A sampling period for obtaining the second gazing point 307 for the second window 203 is referred to as a second period. A sampling period for obtaining the third gazing point for the third window 205 is referred to as a third period. A sampling period for obtaining the fourth gazing point for the fourth window 207 is referred to as a fourth period.

The first period corresponds to the entire period for explaining item A, as indicated by an arrow 1001. However, as indicated by an arrow 1005, the first period may be started from a time point that is a predetermined time T _{1 after the} timing of switching from the description of the item A to the description of the item B.

The second period corresponds to the entire period for explaining item B as indicated by an arrow 1003. However, as indicated by arrows 1007 and 1011, the second period may end when a predetermined time T _{2 has} elapsed from the timing of switching from the description of item A to the description of item B. Further, as indicated by an arrow 1009 and an arrow 1011, the second period may be started from the time when a predetermined transition period T ₃ has passed from the timing of switching from the description of item A to the description of item B.

The third period corresponds to the entire period for explaining item C, as indicated by arrow 1021. However, as indicated by an arrow 1025, the third time period may be started from a timing predated from the timing of switching to the description of item D from the description of item C for a predetermined time T _1.

The fourth period corresponds to the entire period for explaining the item D as indicated by an arrow 1023. However, as indicated by the arrow 1027 and the arrow 1031, the fourth period may end when a predetermined time T _{2 has} elapsed from the timing when the description of the item C is switched to the description of the item D. Further, as indicated by an arrow 1029 and an arrow 1031, the fourth period may be started from the time when a predetermined transition period T ₃ has passed from the timing when the description of the item C is switched to the description of the item D.

Returning to the description of FIG. 9A. When the first period indicated by the arrow 1001 in FIG. 10 is used, the first notification unit 815 notifies the evaluation unit 805 of the start of the first period when the description of the item A is started. On the other hand, when the first period indicated by the arrow 1005 in FIG. 10 is used, the first notification unit 815 is used when the period from the description of item A to the timing of switching to the description of item B reaches a predetermined time T _1. May notify the evaluation unit 805 of the start of the first period.

  The first determination unit 817 determines whether or not it is time to move to the description of the item B (S909). At the point in time when the output of the sound describing the item A is finished, the first determination unit 817 determines that the timing for moving to the description of the item B has been reached.

  When it is determined that the timing for moving to the description of item B has not been reached, the first determination unit 817 stands by until the timing for shifting to the description of item B is reached.

  When it is determined that the timing for moving to the description of item B has been reached, the first notification unit 815 notifies the evaluation unit 805 of the end of the first period (S911). Then, the processing proceeds to S913 illustrated in FIG.

  Turning to the description of FIG. 9B. The display processing unit 811 displays the second window 203 at a predetermined display position on the theme S page (S913). At this time, the brightness of the second window 203 may be increased. Alternatively, an animation of an avatar pointing the second window 203 with a finger may be displayed.

  The audio output unit 813 starts outputting audio that describes the item B (S915). At this time, a voice message instructing to pay attention to the second window 203 may be output at the beginning. Then, the first notification unit 815 notifies the evaluation unit 805 of the start of the second period (S917). That is, when the description of the item B is started, the first notification unit 815 notifies the evaluation unit 805 of the start of the second period. This notification includes the size and position of the second window 203.

  And the 1st determination part 817 determines whether it came to the timing which complete | finishes description of the theme S (S919). At the time when the output of the sound describing the item B is completed, the first determination unit 817 determines that it is time to finish the description of the theme S.

  If it is determined that the timing for finishing the description of the theme S has not been reached, the first determination unit 817 waits until the timing for finishing the description of the theme S is reached.

  If it is determined that it is time to finish the description of the theme S, the first notification unit 815 notifies the evaluation unit 805 of the end of the second period (S921).

  The receiving unit 819 receives a determination result regarding the concentration of the viewer with respect to the theme S from the evaluation unit 805 (S922). Note that determination results regarding the concentration of the viewer with respect to the theme S may be received together in S945 of FIG. Then, the process proceeds to the process of S923 illustrated in FIG.

  Turning to the description of FIG. 9C. At this point, the contents of the explanation are switched from theme S to theme T. The display processing unit 811 displays the theme T page on the screen of the display device 101 (S923).

  The display processing unit 811 displays the third window 205 at a predetermined display position on the theme T page (S925). At this time, the brightness of the third window 205 may be increased. Alternatively, an animation of an avatar pointing the third window 205 with a finger may be displayed.

  The audio output unit 813 starts outputting audio that describes the item C (S927). At this time, a voice message instructing to pay attention to the third window 205 may be output at the beginning. Then, the first notification unit 815 notifies the evaluation unit 805 of the start of the third period (S929). This notification includes the size and position of the third window 205.

When the third period indicated by the arrow 1021 in FIG. 10 is used, the first notification unit 815 notifies the evaluation unit 805 of the start of the third period when the description of the item C is started. On the other hand, in the case of using the third period shown by the arrow 1025 in FIG. 10 is a time period from the description of the item C to the timing of switching to the description of the item D it is, has reached a predetermined time T _1, the first notification unit In step 815, the evaluation unit 805 may be notified of the start of the third period.

  The first determination unit 817 determines whether or not it is time to proceed to the description of the item D (S931). At the point in time when the output of the sound that explains item C is finished, the first determination unit 817 determines that it is time to move to the description of item D.

  If it is determined that the timing for shifting to the description of item D has not been reached, the first determination unit 817 waits until it reaches the timing for shifting to the description of item D.

  When it is determined that the timing for moving to the description of the item D has been reached, the first notification unit 815 notifies the evaluation unit 805 of the end of the third period (S933). Then, the process proceeds to the process of S935 illustrated in FIG.

  Turning to the description of FIG. 9D. The display processing unit 811 displays the fourth window 207 at a predetermined display position on the theme T page (S935). At this time, the brightness of the fourth window 207 may be increased. Alternatively, an animation of an avatar pointing the fourth window 207 with a finger may be displayed.

  The audio output unit 813 starts outputting audio that describes the item D (S937). At this time, a voice message instructing to pay attention to the fourth window 207 may be output at the beginning. Then, the first notification unit 815 notifies the evaluation unit 805 of the start of the fourth period (S939). That is, when the description of the item D is started, the first notification unit 815 notifies the evaluation unit 805 of the start of the fourth period. This notification includes the size and position of the fourth window 207.

  The first determination unit 817 determines whether it is time to finish the description of the theme T (S941). At the time when the output of the sound describing the item D is finished, the first determination unit 817 determines that it is time to finish the description of the theme T.

  When it is determined that the timing for finishing the explanation of the theme T has not been reached, the first determination unit 817 waits until the timing for finishing the explanation of the theme T is reached.

  If it is determined that it is time to finish the description of the theme T, the first notification unit 815 notifies the evaluation unit 805 of the end of the fourth period (S943). Then, the process proceeds to S945 illustrated in FIG.

  Turning to the description of FIG. 9E. The receiving unit 819 receives a determination result regarding the concentration of the viewer with respect to the theme T from the evaluation unit 805 (S945). As described above, the receiving unit 819 may receive the determination result regarding the concentration of the viewer with respect to the theme S at this timing.

  The first determination unit 817 determines whether the viewer has concentrated on the theme S based on the determination result (S947). When it is determined that the viewer is not concentrated on the theme S, the display processing unit 811 displays a test page for the theme S (S949), and the first reception unit 821 receives an answer in the test for the theme S. (S951).

  Upon receiving an answer in the theme S test, the display processing unit 811 displays an explanation page related to the theme S test (S953). When the comment page is closed, the process proceeds to S955. For example, when the viewer gives an instruction to close the page, the comment page is closed.

  On the other hand, in the process of S947, when it is determined that the viewer is concentrated on the theme S, the test on the theme S is not performed. Accordingly, the process proceeds to S955 as it is.

  The first determination unit 817 determines whether the viewer is concentrated on the theme T based on the determination result (S955). If it is determined that the viewer is not concentrated on the theme T, the display processing unit 811 displays the test page for the theme T (S957), and the first receiving unit 821 receives an answer in the test for the theme T. (S959).

  Upon receiving an answer in the theme T test, the display processing unit 811 displays an explanation page related to the theme T test (S961). The presentation process ends when the comment page is closed. For example, when the viewer gives an instruction to close the page, the comment page is closed.

  On the other hand, in the process of S955, when it is determined that the viewer is concentrated on the theme T, the test on the theme T is not performed. Therefore, the presentation process is finished as it is.

  Subsequently, the evaluation process will be described. The evaluation process is executed in parallel with the presentation process. 11A and 11B show an example of the evaluation process flow. The control unit 831 starts measuring the intersection coordinates using the line-of-sight sensor device 103 (S1101). The intersection point coordinates are coordinates pointed to by the viewer's line of sight, that is, coordinates indicating the intersection point of the viewer's line of sight and the screen of the display device 101. The control unit 831 repeats the measurement of the intersection coordinates as needed until the end of measurement is instructed.

  The second reception unit 833 receives a notification of the start of the first period from the presentation unit 803 (S1103). The size and position of the first window 201 received at this time are stored in the layout storage unit 845. Upon receiving the notification of the start of the first period, the specifying unit 835 executes a first specifying process (S1105).

  FIG. 12 shows an example of the first specific processing flow. It is assumed that the frequency data stored in the frequency storage unit 847 has been initialized at the time of starting the first specifying process. That is, the frequency for each pixel is zero.

  The identifying unit 835 acquires the intersection coordinates detected at the current time from the control unit 831 (S1201). Further, the specifying unit 835 specifies the peripheral area 303 of the intersection (S1203). The shape of the peripheral region 303 is the same as that of the first window 201, and the center of gravity of the peripheral region 303 coincides with the intersection.

  The specifying unit 835 adds 1 to the frequency corresponding to each pixel included in the peripheral area 303 (S1205). Then, the specifying unit 835 determines whether or not the second reception unit 833 has received a notification of the end of the first period (S1207).

  If it is determined that the notification of the end of the first period has not yet been received, the processing returns to S1201 and the above-described processing is repeated.

  On the other hand, when determining that the notification of the end of the first period has been received, the specifying unit 835 specifies the maximum frequency in the frequency data (S1209).

  The maximum frequency will be described with reference to FIG. FIG. 13 is an enlarged view of the vicinity of the upper left of the display surface shown in FIG. As described above, the frequency of overlap of the peripheral region 303 at the first gazing point 305 is 4. This corresponds to the maximum frequency. A range 1301 is the same size as the first window 201 when the first gazing point 305 is aligned with the center of gravity. The range 1301 includes intersection points 301a to 301d. Thus, the maximum frequency coincides with the frequency that the viewer's line of sight was within the range 1301 corresponding to the size of the first window 201 in the first period.

  Returning to the description of FIG. Further, the specifying unit 835 specifies a pixel having the highest frequency, and sets a point corresponding to the pixel as the first gazing point (S1211). If there are a plurality of pixels having the highest frequency, one of the pixels is specified. For example, the pixel closest to the center of gravity among the plurality of pixels is specified. When the first specifying process is finished, the process returns to the process of S1107 shown in FIG. 11A.

Returning to the description of FIG. The second reception unit 833 receives a notification of the start of the second period from the presentation unit 803 (S1107). The size and position of the second window 203 received at this time are stored in the layout storage unit 845. Upon receiving the notification of the start of the second period, the specifying unit 835 executes a second specifying process (S1109). When the period of the arrow 1009 or the arrow 1011 shown in FIG. 10 is used as the second period, the specifying unit 835 receives the notification of the start of the second period, and when the transition period T ₃ has elapsed, Processing may be executed.

The second specifying process is substantially the same as the first specifying process. The shape of the peripheral area 303 in the second specifying process is equal to that of the second window 203. In S1207 shown in FIG. 12, it is determined whether or not a notification of the end of the second period has been received instead of the notification of the end of the first period. Also, in the case of the second period the duration of arrow 1007 or arrow 1011 shown in FIG. 10, like in S1207, it determines whether after receiving the notification of the first period ends a predetermined time T ₂ has elapsed To do. Similarly, in S1211, the pixel with the highest frequency is set as the second gazing point.

  The calculation unit 837 calculates the first vector 405 (S1111). The first vector 405 is a vector having a reference point 401 in the first window 201 as a start point and a reference point 403 in the second window 203 as an end point. Therefore, the first vector 405 is calculated by subtracting the coordinates of the reference point 401 in the first window 201 from the coordinates of the reference point 403 in the second window 203.

  Note that the coordinates of the reference point 401 in the first window 201 are set such that the half of the width of the first window 201 is added to the X coordinate indicating the position of the first window 201, and the first window is set to the Y coordinate indicating the position of the first window 201. It is determined by adding half the height of 201. Similarly, the coordinates of the reference point 403 in the second window 203 are obtained by adding half the width of the second window 203 to the X coordinate indicating the position of the second window 203, and adding the second coordinate to the Y coordinate indicating the position of the second window 203. It is determined by adding half the height of the window 203.

  The calculating unit 837 calculates the second vector 309 (S1113). The second vector 309 is a vector having the first gazing point 305 as a start point and the second gazing point 307 as an end point. Accordingly, the second vector 309 is calculated by subtracting the coordinates of the first gazing point 305 specified by the first specifying process from the coordinates of the second gazing point 307 specified by the second specifying process.

  The calculating unit 837 calculates the degree of approximation between the direction of the first vector 405 and the direction of the second vector 309 (S1115). In this example, the degree of approximation between the direction of the first vector 405 and the direction of the second vector 309 is a cosine value cos θ of the angle θ formed by the first vector 405 and the second vector 309. For example, by calculating the inner product of the first vector 405 and the second vector 309 and dividing the inner product value by the magnitude of the first vector 405 and the magnitude of the second vector 309, the cosine value cos θ is obtained. Also good.

  The calculating unit 837 calculates the index value of the degree of concentration from the first period to the second period (S1117). The calculation unit 837 calculates the maximum frequency obtained in the second specifying process, that is, the frequency that the viewer's line of sight was within the range 1301 corresponding to the size of the second window 203 in the second period. The frequency (per time) at which the second window 203 is watched is obtained by dividing by the above. Then, the index value of the degree of concentration is obtained by the degree of approximation between the direction of the first vector 405 and the direction of the second vector 309 + coefficient × frequency of watching the second window 203 (per hour). When the coefficient is 0 in this expression, the index value of the degree of concentration is equal to the degree of approximation between the direction of the first vector 405 and the direction of the second vector 309. According to this formula, the index value of the degree of concentration indicates that the viewer's line of sight followed from the first window 201 to the second window 203 and that the time spent gazing within the range of the second window 203 was long. Reflects.

  The second determination unit 839 determines whether or not the viewer has concentrated on the content from the first period to the second period (S1119). The second determination unit 839 determines that the viewer has concentrated on the content from the first period to the second period when the index value of the degree of concentration is equal to or greater than the threshold value. On the contrary, the second determination unit 839 determines that the viewer has not concentrated on the content from the first period to the second period when the index value of the degree of concentration does not exceed the threshold value.

  The second notification unit 841 passes the determination result regarding the concentration on the content from the first period to the second period to the presentation unit 803 (S1121). In addition, at the timing of S1139 in FIG. 11B to be described later, the determination result regarding the concentration to the content from the first period to the second period is presented together with the determination result regarding the concentration to the content from the third period to the fourth period. You may make it pass to the part 803. That is, each time determination regarding concentration on content is performed, the determination result may be separately delivered to the presentation unit 803, or each determination result may be collectively delivered to the presentation unit 803. Then, the processing shifts to the processing of S1123 illustrated in FIG.

  Turning to the description of FIG. 11B. The second reception unit 833 receives a notification of the start of the third period from the presentation unit 803 (S1123). The size and position of the third window 205 received at this time are stored in the layout storage unit 845. When the notification of the start of the third period is received, the specifying unit 835 executes the third specifying process (S1125).

  The third specifying process is substantially the same as the first specifying process. The shape of the peripheral area 303 in the third specifying process is equal to that of the third window 205. In S1207 illustrated in FIG. 12, it is determined whether or not a notification of the end of the third period has been received instead of the notification of the end of the first period. Similarly, in S1211, the pixel with the highest frequency is set as the third gazing point.

The second reception unit 833 receives a notification of the start of the fourth period from the presentation unit 803 (S1127). The size and position of the fourth window 207 received at this time are stored in the layout storage unit 845. The specifying unit 835 executes the fourth specifying process (S1129). When the period indicated by the arrow 1029 or the arrow 1031 illustrated in FIG. 10 is used as the fourth period, the specifying unit 835 receives the notification of the start of the fourth period, and when the transition period T ₃ has elapsed, Processing may be executed.

The fourth specifying process is substantially the same as the first specifying process. The shape of the peripheral area 303 in the fourth specifying process is the same as that of the fourth window 207. In S1207 shown in FIG. 12, it is determined whether or not a fourth period end notification is received instead of the first period end notification. Further, when the period of the fourth period of arrow 1027 or arrow 1031 shown in FIG. 10, like in S1207, determines whether after receiving the notification of the third period ends a predetermined time T ₂ has elapsed To do. Similarly, in S1211, the pixel with the highest frequency is set as the fourth gazing point.

  The calculating unit 837 calculates the third vector (S1131). The third vector is a vector having a reference point in the third window 205 as a start point and a reference point in the fourth window 207 as an end point. Accordingly, the third vector is calculated by subtracting the reference point coordinates in the third window 205 from the reference point coordinates in the fourth window 207.

  Note that the coordinates of the reference point 401 in the third window 205 are set such that the half of the width of the third window 205 is added to the X coordinate indicating the position of the third window 205, and the third window is set to the Y coordinate indicating the position of the third window 205. It is determined by adding half the height of 205. Similarly, the coordinate of the reference point 403 in the fourth window 207 is the fourth coordinate in the Y coordinate indicating the position of the fourth window 207 by adding half the width of the fourth window 207 to the X coordinate indicating the position of the fourth window 207. It is determined by adding half the height of the window 207.

  The calculating unit 837 calculates the fourth vector (S1133). The fourth vector is a vector having a third gazing point as a start point and a fourth gazing point as an end point. Therefore, the fourth vector is calculated by subtracting the coordinates of the third gazing point specified by the third specifying process from the coordinates of the fourth gazing point specified by the fourth specifying process.

  The calculating unit 837 calculates the degree of approximation between the direction of the third vector and the direction of the fourth vector (S1135). In this example, the degree of approximation between the direction of the third vector and the direction of the fourth vector is a cosine value cos γ of the angle γ formed by the third vector and the fourth vector. For example, the inner product of the third vector and the fourth vector may be obtained, and the value of the cosine may be obtained by dividing the value of the inner product by the magnitude of the third vector and the magnitude of the fourth vector.

  The calculation unit 837 calculates the index value of the degree of concentration from the third period to the fourth period (S1137). The calculation unit 837 calculates the maximum frequency obtained in the fourth specifying process, that is, the frequency that the viewer's line of sight was within the range corresponding to the size of the fourth window 207 in the fourth period. The frequency (per time) at which the fourth window 207 is watched is obtained. Then, the index value of the degree of concentration is calculated according to the following formula. The degree of approximation of the direction of the third vector and the direction of the fourth vector + coefficient × frequency of watching the fourth window 207 (per time). When the coefficient is 0 in this equation, the index value of the degree of concentration is equal to the degree of approximation between the direction of the third vector and the direction of the fourth vector. According to this expression, the index value of the degree of concentration indicates that the viewer's line of sight followed from the third window 205 to the fourth window 207 and that the time spent gazing within the range of the fourth window 207 was long. Reflects.

  The second determination unit 839 determines whether or not the viewer has concentrated on the content from the third period to the fourth period (S1138). The second determination unit 839 determines that the viewer has concentrated on the content from the third period to the fourth period when the index value of the degree of concentration is equal to or greater than the threshold. Conversely, the second determination unit 839 determines that the viewer has not concentrated on the content from the third period to the fourth period when the index value of the degree of concentration does not exceed the threshold value.

  The second notification unit 841 passes the determination result regarding the concentration on the content from the third period to the fourth period to the presentation unit 803 (S1139). As described above, at this time, the determination result regarding the concentration on the content from the first period to the second period may also be passed to the presentation unit 803.

  The control unit 831 stops the measurement of the intersection coordinates using the line-of-sight sensor device 103 (S1141). Then, the output unit 843 outputs the index value of the degree of concentration (S1143). For example, in association with each period to be evaluated (in this example, the first period to the second period (theme S explanation period) and the third period to the fourth period (theme T explanation period)), The index value of the degree of concentration in the period is output. The output unit 843 may transmit the index value of the degree of concentration to the teacher terminal, for example. Or you may make it the output part 843 transmit the index value of a concentration degree to a viewer's terminal, for example. Further, the output unit 843 may attach a determination result (whether or not it is concentrated) to the index value of the degree of concentration. The output unit 843 may add a statistical value (for example, an average value and a variance) regarding the index value of the degree of concentration to the index value of the degree of concentration.

  According to the present embodiment, it is possible to more accurately grasp the degree of viewer concentration with respect to the window displayed on the screen.

  In addition, it is possible to reflect the degree of attention focused on the previous window whose line of sight has been shifted to the index value indicating the degree of concentration of the viewer.

  In addition, the display content of the content can be changed according to the degree of concentration on the content.

  Further, it is possible to more accurately grasp the degree of concentration of the viewer with respect to the voice.

[Embodiment 2]
In this embodiment, another window display mode will be described.

  FIG. 14 shows an example of screen transition according to the second embodiment. The first level in FIG. 14 is the same as the first level in FIG. In the second embodiment, as shown in the second row of FIG. 14, the first window 201 disappears when the second window 203 is displayed. The third level in FIG. 14 is the same as the third level in FIG. In the second embodiment, as shown in the fourth row in FIG. 14, the third window 205 disappears when the fourth window 207 is displayed.

  The presentation process according to Embodiment 2 will be described with reference to FIGS. 15A and 15B. First, similarly to the case of the first embodiment, the processing shown in S901 to S911 of FIG. 9A is performed. Then, the process proceeds to S1501 illustrated in FIG.

  The display processing unit 811 deletes the first window 201 from the theme S page (S1501). The processing from S913 to S922 shown in FIG. 15A is the same as that in FIG. 9B. Then, the process proceeds to the process of S923 illustrated in FIG.

  As in the case of Embodiment 1, the processing shown in S923 to S933 in FIG. 9C is performed. Then, the process proceeds to the process of S1503 illustrated in FIG.

  The display processing unit 811 deletes the third window 205 from the theme T page (S1503). The processing from S935 to S943 shown in FIG. 15B is the same as that in FIG. 9D. Then, the process proceeds to S945 illustrated in FIG.

  As in the case of the first embodiment, the processing shown in S945 to S961 in FIG. 9E is performed.

  According to the present embodiment, the switching of windows becomes clearer.

[Embodiment 3]
In the present embodiment, another window display mode will be described.

  FIG. 16 shows an example of screen transition according to the third embodiment. In the third embodiment, as shown in the first row of FIG. 16, when the first window 201 is displayed, the second window 203 is also displayed. As shown in the second row in FIG. 16, the display layout does not change at the timing when the description shifts to item B. In the third embodiment, as shown in the third row in FIG. 16, when the third window 205 is displayed, the fourth window 207 is also displayed. Then, as shown in the fourth row of FIG. 16, the display layout does not change at the timing when the description shifts to the item D.

  A presentation process according to the third embodiment will be described with reference to FIGS. 17A to 17D. The processes shown in S901 and S903 in FIG. 17A are the same as those in FIG. 9A. The display processing unit 811 displays the second window 203 at a predetermined display position on the theme S page (S1701).

  The processing shown in S905 and S911 in FIG. 17A is the same as that in FIG. 9A. Then, the process proceeds to the process of S915 illustrated in FIG.

  Turning to FIG. 17B. The processing from S915 to S922 shown in FIG. 17B is the same as that in FIG. 9B. Then, the process proceeds to the process of S923 illustrated in FIG.

  Turning to the description of FIG. 17C. The processing shown in S923 and S925 in FIG. 17C is the same as that in FIG. 9C. The display processing unit 811 displays the fourth window 207 at a predetermined display position in the theme T page (S1703).

  The processing shown in S927 and S933 in FIG. 17C is the same as that in FIG. 9C. Then, the process proceeds to the process of S937 illustrated in FIG.

  Turning to the description of FIG. 17D. The processing from S937 to S943 shown in FIG. 17D is the same as that in FIG. 9D. Then, the process proceeds to S945 illustrated in FIG.

  As in the case of the first embodiment, the processing shown in S945 to S961 in FIG. 9E is performed.

  According to the present embodiment, the window display process becomes simpler.

[Embodiment 4]
In this embodiment, another feedback mode will be described.

  The presentation process according to the fourth embodiment will be described with reference to FIGS. First, similarly to the case of the first embodiment, the processing of S901 to S922 shown in FIGS. 9A and 9B is executed. Then, the process proceeds from the terminal B shown in FIG. 9B to the process shown in S1801 via the terminal B1 shown in FIG.

  The receiving unit 819 receives the determination result regarding the concentration of the viewer with respect to the theme S (S1801). The first determination unit 817 determines whether the viewer is concentrated on the theme S based on the determination result (S1803).

  If it is determined that the viewer is not concentrated on the theme S, the display processing unit 811 displays a caution window (S1805). In the caution window, a caution comment that prompts the viewer to concentrate on the content is displayed. The first determination unit 817 determines whether or not the attention window has been closed (S1807). The display processing unit 811 may automatically close the caution window when a predetermined time has elapsed. If it is determined that the caution window has not yet been closed, the first determination unit 817 waits until the timing for closing the caution window is reached.

  If it is determined in S1807 that the caution window has been closed, the process proceeds from the terminal B2 to the process in S923 via the terminal B illustrated in FIG. 9C.

  Returning to the description of S1803, when it is determined that the viewer is concentrated on the theme S, the process proceeds from the terminal B2 to the process of S923 via the terminal B shown in FIG. 9C.

  As in the case of the first embodiment, the processing of S923 to S943 shown in FIGS. 9C and 9D is executed. Then, the process proceeds from the terminal D shown in FIG. 9D to the process shown in S1901 via the terminal D shown in FIG.

  The receiving unit 819 receives the determination result regarding the concentration of the viewer with respect to the theme T (S1901). The first determination unit 817 determines, based on the determination result, whether or not the viewer has concentrated on the theme T (S1903).

  If it is determined that the viewer is not concentrated on the theme T, the display processing unit 811 displays a caution window (S1905). The first determination unit 817 determines whether or not the attention window has been closed (S1907). If it is determined that the caution window has not yet been closed, the first determination unit 817 waits until the timing for closing the caution window is reached.

  If it is determined in S1907 that the caution window has been closed, the presentation process ends.

  Returning to the description of S1903, the presentation process is also ended when it is determined that the viewer is concentrated on the theme T.

  According to the present embodiment, it is possible to urge the viewer to concentrate on the content.

[Embodiment 5]
In the present embodiment, another feedback aspect will be described.

  The presentation process according to the fifth embodiment will be described with reference to FIGS. First, similarly to the case of the first embodiment, the processing of S901 to S922 shown in FIGS. 9A and 9B is executed. Then, the process proceeds to the process shown in S2001 from the terminal B shown in FIG. 9B via the terminal B3 shown in FIG.

  The receiving unit 819 receives a determination result regarding the concentration of the viewer with respect to the theme S (S2001). The first determination unit 817 determines whether the viewer has concentrated on the theme S based on the determination result (S2003).

  If it is determined that the viewer is concentrated on the theme S, the display processing unit 811 displays a supplementary window for the theme S (S2005). In the supplement window, content supplementing the theme S (for example, a small story about the theme S) is displayed. The first determination unit 817 determines whether or not the supplementary window is closed (S2007). For example, the supplementary window may be closed when the first reception unit 821 receives a window closing instruction. If it is determined that the supplementary window has not yet been closed, the first determination unit 817 waits until the timing for closing the supplementary window is reached.

  If it is determined in S2007 that the supplementary window is closed, the process proceeds from the terminal B4 to the process of S923 via the terminal B illustrated in FIG. 9C.

  Returning to the description of S2003, even when it is determined that the viewer is not concentrated on the theme S, the process proceeds from the terminal B4 to the process of S923 via the terminal B shown in FIG. 9C.

  As in the case of the first embodiment, the processing of S923 to S943 shown in FIGS. 9C and 9D is executed. Then, the process proceeds from the terminal D illustrated in FIG. 9D to the process illustrated in S2101 via the terminal D illustrated in FIG.

  The receiving unit 819 receives the determination result regarding the concentration of the viewer with respect to the theme T (S2101). The first determination unit 817 determines whether the viewer has concentrated on the theme T based on the determination result (S2103).

  If it is determined that the viewer is concentrated on the theme T, the display processing unit 811 displays a supplemental window for the theme T (S2105). In the supplement window, contents supplementing the theme T (for example, a small story about the theme T) are displayed. The first determination unit 817 determines whether or not the supplementary window is closed (S2107). If it is determined that the supplementary window has not yet been closed, the first determination unit 817 waits until the timing for closing the supplementary window is reached.

  If it is determined in step S2107 that the supplementary window has been closed, the presentation process ends.

  Returning to the description of S2103, the presentation process is also ended when it is determined that the viewer has not concentrated on the theme T.

  According to this embodiment, it is possible to respond to the interest of the viewer.

  The configuration and processing of the learning system described above may be applied to an advertising system. In the advertisement system, the theme S may be replaced with the introduction of the product S. In that case, the item A may be replaced with the concept of the product S, and the item B may be replaced with word-of-mouth information for the product S. Similarly, the theme T may be replaced with the introduction of the product T. In this case, the item C may be replaced with the concept of the product T, and the item D may be replaced with word-of-mouth information for the product T.

  Moreover, you may replace the test regarding the theme S with the questionnaire which requests | requires the request with respect to the goods S. FIG. Similarly, the test related to the theme T may be replaced with a questionnaire for requesting the product T.

  Further, the supplement window of the theme S may be replaced with a price window that presents the price of the product S. Similarly, the supplemental window of theme T may be replaced with a price window that presents the price of product T.

  Furthermore, you may apply the structure and process of the learning system which were mentioned above to the menu system of a restaurant. In the menu system, the theme S may be replaced with the introduction of the dish S. In that case, the item A may be replaced with an image of the cooking process of the dish S, and the item B may be replaced with an image on which the dish S is arranged. Similarly, the theme T may be replaced with the introduction of the dish T. In that case, the item C may be replaced with an image of the cooking process of the dish T, and the item D may be replaced with an image on which the dish T is arranged.

  The supplementary window for theme S may be replaced with an order window for ordering food S. Similarly, the supplementary window for theme T may be replaced with an order window for ordering dishes T.

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the functional block configuration described above may not match the program module configuration.

  Further, the configuration of each storage area described above is an example, and the above configuration is not necessarily required. Further, in the processing flow, if the processing result does not change, the processing order may be changed or a plurality of processes may be executed in parallel.

  The information processing device 801 described above is a computer device, and as shown in FIG. 22, a memory 2501, a CPU (Central Processing Unit) 2503, a hard disk drive (HDD: Hard Disk Drive) 2505, and a display device. A display control unit 2507 connected to 2509, a drive device 2513 for the removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for executing the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In the embodiment of the present invention, an application program for performing the above-described processing is stored in a computer-readable removable disk 2511 and distributed, and installed in the HDD 2505 from the drive device 2513. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above and programs such as the OS and application programs. .

  The embodiment of the present invention described above is summarized as follows.

  In the evaluation method according to the present embodiment, (A) the first gaze point of the user in the first period in which at least the first object is displayed on the screen is specified, and (B) at least the second object after the first period. The second gaze point of the user in the second period is displayed in the screen, and (C) the direction of the first vector from the reference point in the first object to the reference point in the second object, and the first It includes processing for calculating the degree of approximation with the direction of the second vector from the gazing point to the second gazing point.

  In this way, the user's degree of concentration with respect to the display object can be grasped more accurately.

  Furthermore, the frequency per time when the user's line of sight was within the range corresponding to the size of the second object in the second period may be calculated. An index value indicating the degree of user concentration may be calculated based on the degree of approximation and the frequency per time.

  In this way, it is possible to reflect the degree of attention paid to the previous object whose line of sight has been shifted to the index value indicating the degree of user concentration.

  Furthermore, it may be determined based on at least the degree of approximation whether or not the user has concentrated on the content including the first object and the second object. Further, content display may be controlled in accordance with the determination result.

  In this way, the display content of the content can be changed according to the degree of concentration on the content.

  Furthermore, you may make it output the audio | voice regarding a 1st object in a 1st period. Moreover, you may make it output the audio | voice regarding a 2nd object in a 2nd period.

  In this way, the user's degree of concentration on the voice can be grasped more accurately.

  A program for causing a computer to perform the processing according to the above method can be created. The program can be a computer-readable storage medium such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, a hard disk, or the like. It may be stored in a storage device. Note that intermediate processing results are generally temporarily stored in a storage device such as a main memory.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1)
Identifying at least the first gaze point of the user in the first period of displaying the first object on the screen;
After the first period, specify a second gaze point of the user in a second period in which at least a second object is displayed in the screen,
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point An evaluation method that includes a process and is executed by a computer.

(Appendix 2)
Furthermore,
Calculating the frequency per time that the user's line of sight was within a range corresponding to the size of the second object in the second period;
The evaluation method according to claim 1, further comprising: calculating an index value indicating the degree of user concentration based on the degree of approximation and the frequency per time.

(Appendix 3)
Furthermore,
Based on at least the degree of approximation, it is determined whether or not the user has concentrated on content including the first object and the second object;
The evaluation method according to appendix 1 or 2, including a process of controlling display of the content according to a determination result.

(Appendix 4)
Furthermore,
Outputting a sound related to the first object in the first period;
The evaluation method according to any one of appendices 1 to 3, including a process of outputting a sound related to the second object in the second period.

(Appendix 5)
Identifying at least the first gaze point of the user in the first period of displaying the first object on the screen;
After the first period, specify a second gaze point of the user in a second period in which at least a second object is displayed in the screen,
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point Yes A program that causes a computer to execute processing.

(Appendix 6)
A first gaze point of a user in a first period in which at least a first object is displayed in the screen is specified, and after the first period, the user's first glance in a second period in which at least a second object is displayed in the screen. 2 specific part to identify the gaze point;
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point An evaluation device comprising:

DESCRIPTION OF SYMBOLS 101 Display apparatus 102 Speaker 103 Line-of-sight sensor apparatus 201 1st window 203 2nd window 205 3rd window 207 4th window 301 Intersection 303 Peripheral area 305 1st gazing point 307 2nd gazing point 309 2nd vector 401 Reference point 403 Reference point 405 First Vector 801 Information Processing Device 803 Presentation Unit 805 Evaluation Unit 811 Display Processing Unit 813 Audio Output Unit 815 First Notification Unit 817 First Determination Unit 819 Receiving Unit 821 First Accepting Unit 823 Content Storage Unit 831 Control Unit 833 Second Reception unit 835 Identification unit 837 Calculation unit 839 Second determination unit 841 Second notification unit 843 Output unit 845 Layout storage unit 847 Frequency storage unit 1301 Range

Claims (5)

Identifying at least the first gaze point of the user in the first period of displaying the first object on the screen;
After the first period, specify a second gaze point of the user in a second period in which at least a second object is displayed in the screen,
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point An evaluation method that includes a process and is executed by a computer. Furthermore,
Calculating the frequency per time that the user's line of sight was within a range corresponding to the size of the second object in the second period;
The evaluation method according to claim 1, further comprising: calculating an index value indicating the degree of user concentration based on the degree of approximation and the frequency per time. Furthermore,
Based on at least the degree of approximation, it is determined whether or not the user has concentrated on content including the first object and the second object;
The evaluation method according to claim 1, further comprising a process of controlling display of the content according to a determination result. Identifying at least the first gaze point of the user in the first period of displaying the first object on the screen;
After the first period, specify a second gaze point of the user in a second period in which at least a second object is displayed in the screen,
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point Yes A program that causes a computer to execute processing. A first gaze point of a user in a first period in which at least a first object is displayed in the screen is specified, and after the first period, the user's first glance in a second period in which at least a second object is displayed in the screen. 2 specific part to identify the gaze point;
Approximation between the direction of the first vector from the reference point in the first object to the reference point in the second object and the direction of the second vector from the first gazing point to the second gazing point An evaluation device comprising:

Images