JP2022135476A - Information processing apparatus and program - Google Patents

Abstract

To efficiently grasp a mental state of a participant in a lecture in an on-demand format, which is an important issue when a provider of the lecture maintains and improves quality of a service.SOLUTION: A lecturer-side information acquisition unit 101 acquires lecturer-side information on a lecturer T who transmits information while looking at a resume R displayed in at least a part of a screen of a lecturer terminal 2, the lecturer-side information indicating a correspondence relationship between the lecturer's line-of-sight Lt and the resume R. A participant-side information acquisition unit 102 acquires participant-side information on a participant S who receives the transmission of information while looking at the resume R displayed in at least a part of a screen of a participant terminal 3, the participant-side information indicating the correspondence relationship between the participant's line-of-sight Ls and the resume R. A detection unit 103 detects feature quantities of the lecturer-side information and the participant-side information. An estimation unit 104 estimates a mental state of the participant S on the basis of, the detected feature quantities.SELECTED DRAWING: Figure 12

Description

The present invention relates to an information processing device and a program.

People's learning environments are diversifying with the recent changes in social conditions. Among them, expectations are rising for online learning, which enables learning to continue in response to rapid changes in social conditions, and related technologies for supporting online learning have been proposed (see, for example, Patent Document 1).
In addition, so-called on-demand lectures that can be taken by reproducing the contents of pre-recorded lectures are an example of online learning, and the importance of their existence is increasing day by day.

JP-A-2003-330355

However, in an on-demand lecture, it is of course impossible to even see the student at the time of recording, so it is difficult to grasp the inner state of the student, which can be grasped in a face-to-face lecture. Understanding the internal state of the students who take the lectures is an important issue in maintaining and improving the quality of the service. There was no choice but to guess the internal state of the student from the results of the test that was conducted for the student after the fact.

The present invention has been made in view of this situation, and it is an important issue for the provider of on-demand lectures to maintain and improve the quality of the service. The purpose is to enable the status to be grasped efficiently.

In order to achieve the above object, an information processing device according to one aspect of the present invention includes:
a first acquisition means for acquiring first information indicating a correspondence relationship between a history of movement of a line of sight of a first user who transmits information while viewing a predetermined image displayed at a predetermined position on a first screen and the predetermined image; ,
Acquiring second information indicating a correspondence relationship between a history of movement of the line of sight of a second user to whom the information is transmitted and the predetermined image while viewing the predetermined image displayed at a predetermined position on the second screen; 2 acquisition means;
detection means for detecting the feature amount of the first information and the second information;
estimating means for estimating the internal state of the second user based on the detected feature quantity;
Prepare.

A program of one aspect of the present invention is a program corresponding to the above-described information processing apparatus of one aspect of the present invention.

According to the present invention, the on-demand lecture provider can efficiently grasp the internal state of the lecture attendee, which is an important issue in maintaining and improving the quality of the service. can be done.

1 is a diagram showing an overview of this service that can be realized by an information processing system to which a server according to an embodiment of the invention is applied; FIG. It is a figure which shows the specific example of the screen displayed on a student terminal during taking a lecture. It is a graph which shows the distance between eyes|visual_axis of a student and a lecturer. FIG. 4 is a graph showing the line-of-sight distance between another student different from that in FIG. 3 and the lecturer; FIG. It is a graph which shows the relationship between the movement speed of a lecturer's line of sight, the movement speed of a student's line of sight, and the distance between eyes. (A) is a graph showing the change over time of the distance between the lines of sight when line-of-sight synchronization occurs. (B) is a graph showing the change over time of the eye-to-eye distance in a scene where the student looks at the resume but the eye-gaze non-synchronization occurs. FIG. 7 is a histogram showing the distribution of distances between sight lines between the graph of FIG. 6(A) and the graph of FIG. 6(B); FIG. (A) and (B) are graphs showing the distribution of distances between gazes in other resumes that were not used in the analysis. 1 is a diagram showing an overview of this service that can be realized by an information processing system to which a server according to an embodiment of the invention is applied; FIG. It is a figure showing an example of composition of an information processing system in which a server concerning one embodiment of an information processor of the present invention is applied. 11 is a block diagram showing an example of a hardware configuration of a server in the information processing system shown in FIG. 10; FIG. FIG. 11 is a functional block diagram showing an example of a functional configuration for executing an internal state estimation process among the functional configurations of a server configuring the information processing system of FIG. 10;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, with reference to FIGS. 1 to 9, an outline of a service (hereinafter referred to as "this service") that can be realized by an information processing system to which a server 1 according to an embodiment of the present invention is applied will be described. .

FIG. 1 is a diagram showing an overview of this service that can be realized by an information processing system to which a server according to one embodiment of the present invention is applied.

This service is a service provided to an on-demand lecture provider (not shown) who prepares a lecture of a lecturer T as on-demand content and provides it to a student S. FIG.
In this service, in an on-demand lecture, the internal state of the student S, which is expressed as facial expressions, postures, and actions, is detected based on the information regarding the line of sight of the lecturer T and the student S, respectively. Then, by providing the information to the lecturer T and the on-demand lecture provider, it is possible to support the appropriate approach to the student S. Here, the "internal state" refers to the state of the student S' feelings and interest in the lecture.

This service focuses on eye interaction, which is said to more deeply express the content of a person's thoughts than a person's facial expressions or physical behavior. Specifically, the line of sight is extracted from the captured images of the faces of the lecturer T and the student S, respectively, using ICT (Information and Communication Technology). Then, each of the instructor T and the student S extracts the line of sight of the lecture resume R displayed so as to be shared on at least a part of the screens of the instructor terminal 2 and the student terminal 3, respectively. The extracted line-of-sight history is displayed on the screen as a trajectory. Specifically, on the screen of the lecturer terminal 2 in FIG. 1, a resume R that is displayed so as to be shared and the lecturer's line of sight Lt that shows the movement history of the lecturer T's line of sight in the form of a trajectory are displayed. Also, on the screen of the student terminal 3 in FIG. 1, the resume R displayed so as to be shared and the student's line of sight Ls showing the history of the movement of the student S's line of sight in the form of a trajectory are displayed.

When the on-demand lecture provider creates the contents of the on-demand lecture, the lecturer T starts the lecture while referring to the resume R displayed on at least a part of the screen of the lecturer terminal 2 . At this time, the resume R displayed on the lecturer terminal 2 and the lecturer's line of sight Lt are recorded as lecturer-side information in association with the elapsed time from the start of the lecture.

When the combination of the recorded speech data of the lecture and the data of the resume R is provided to the student S as the content of the on-demand lecture, the student S downloads the content to the student terminal 3. to play. Then, the resume R is displayed on the student terminal 3, and the lecture by the lecturer T begins. A student S takes an on-demand lecture by listening to the content of the lecture given by the lecturer T while looking at the resume R displayed on the student terminal 3 . At this time, the resume R and the student's line of sight Ls displayed on the student terminal 3 are recorded as student-side information in association with the time that has elapsed since the start of the lecture.

In this service, the recorded instructor side information and student side information are compared. By this comparison, it is possible to grasp over time what the lecturer T is looking at during the lecture and what the student S is looking at while attending the lecture. As a result of the comparison, if, for example, the student's line of sight Ls follows the lecturer's line of sight Lt, it can be said that the phenomenon of line-of-sight synchronization has occurred.

First, the principle of line-of-sight synchronization will be described. "Gaze Synchronization" refers to a phenomenon in which the process of "explaining the content of the lecture" indicated by the instructor's line of sight Lt and the process of "understanding the content of the lecture" indicated by the student's line of sight Ls are synchronized. . If the lecturer T gives an explanation while looking at a predetermined position of the resume R and the student S looks at the same resume R while listening seriously, the lecturer T and the student S will follow the same position of the resume R as the lecture progresses. I will look at it step by step. In such a case, line-of-sight synchronization becomes red. The occurrence of line-of-sight synchronization indicates the matching of internal states regarding learning between the instructor and the student.

Gaze synchronization is generally recognized as a phenomenon that occurs when the lecturer and students share screens and slides in online live lectures and face-to-face lectures. In this service, even in an on-demand lecture in which there is a time gap between the timing at which the lecturer T records the lecture content and the timing at which the student S views the lecture content, the student S commits to the lecture. If so, the occurrence of line-of-sight alignment can be detected. In addition, on-demand lectures are premised on the presence of a large number of students S for one lecturer T, but in this service, since student-side information of a plurality of students S is acquired, individual Occurrence of gaze synchronization of the student S can be detected. As a result, it becomes possible to estimate the internal state of each student S in various on-demand lectures of 1:n (where n is an integer value of 1 or more).

Next, the calculation principle of line-of-sight synchronization will be described. Recent advances in ICT technology and AI (artificial intelligence) technology have enabled real-time line-of-sight detection based on image data captured by a camera. As a result, as shown in FIG. 1, the occurrence of line-of-sight synchronization is detected by extracting the lecturer's line of sight Lt when the lecturer T recorded the lecture and the student's line of sight Ls when the student S took the lecture (during playback). can do. Specific steps will be described below.

First, content for an on-demand lecture is created by an on-demand lecture provider. Specifically, the lecturer T explains while intentionally looking at the resume R displayed on at least a part of the screen of the lecturer terminal 2 . At this time, the instructor's line of sight Lt, the history of the instructor's T body movements, and the speech data of the instructor's explanation are recorded (step S1). Specifically, the resume R used by the lecturer T is recorded, and from the data of the image of the lecturer T, the lecturer's line of sight Lt and the movement of the lecturer T's body are recorded. Then, the lecturer's line of sight Lt and the recording of the lecturer's T body movements are analyzed after the lecture is recorded. As a result of the analysis, the timing at which line-of-sight synchronization occurs because lecturer T is gazing at resume R during the lecture and the line-of-sight synchronization does not occur because lecturer T is not looking at resume R. Used when timing is extracted.

Here, "at least part of the screen" means that the resume R is not necessarily displayed on the entire screen of the lecturer terminal 2, but is displayed at an arbitrary position on the so-called desktop screen in an arbitrary size. Because you can. This point is the same when the resume R is displayed on at least a part of the student terminal 3, which will be described later.
In addition, the reason why the word "intentionally" is used is that, depending on the lecturer T, there are cases where the lecture is given without looking at the resume R, and in such a case, the purpose of line-of-sight synchronization is lost. . For this reason, when the content of an on-demand lecture is created, it is necessary to have the lecturer T give the lecture while intentionally viewing the resume R.

A student S attends a lecture by a lecturer T while looking at a resume R displayed on at least a part of the student terminal 3 . At this time, the student's line of sight Ls is recorded (step S2). Specifically, the feature amount is extracted from the moving image data recorded by capturing the face and upper body of the student S using the imaging function of the student terminal 3 . In this case, the line of sight, posture, etc. of the student S are continuously extracted, and the occurrence of line-of-sight synchronization is detected.

When the instructor's line of sight Lt and the student's line of sight Ls are recorded, the degree of synchronization between the two is calculated (step S3).
Since both the lecturer T and the student S are looking at the same resume R, they stare at a predetermined part of the resume R explained by the lecturer T and speak, and the student S listens to the explanation and looks at the predetermined part of the resume R. If they are staring at each other, there is a high probability that the instructor T and the student S are looking at the same part of the resume R with their respective line of sight. Therefore, in this service, the instructor's line of sight Lt and the student's line of sight Ls are displayed superimposed on the resume R, and the moving distance between the two (hereinafter referred to as "distance between the lines of sight") is calculated. As a result, it can be expected that the line-of-sight distance will be within the range of line-of-sight fluctuation and measurement error.

This service includes not only a service for detecting line-of-sight synchronization in on-demand lecture content newly created by recording the line-of-sight Lt of the lecturer, but also a service that does not use the line-of-sight Lt of the lecturer. For example, by recording the line of sight Ls of a plurality of students seriously taking an existing on-demand lecture in which the lecturer's line of sight Lt is not recorded, and specifying the part of the resume R that the student S should see, There is service. In this service for students S, a plurality of students S take an existing on-demand lecture and record the lines of sight Ls of the students at that time. As a result, if the n students S are seriously attending the lecture, most of the student's line of sight Ls of the n students S is superimposed on the resume R or displayed at a position close to it. should be The position of the student's line of sight Ls of the n students S at this time is the position that the student S should look at at each timing in the flow of the explanation of the lecture.

Looking at the result of recording the line of sight of each of the n students S, the lines of sight of each of the n students S may be widely dispersed. In this case, the context of the lecture at that timing is presumed not to look at a specific part of the resume R, but to look at parts of n students S who have different interests and interpretations. In this case, the mutual distance between the lines of sight of the n students S at that timing is of low importance as information for line-of-sight synchronization. On the other hand, the mutual distance between the sight lines of the n students S at the timing when the dispersion of the sight lines is small is evaluated as information with higher importance.

In addition, some of the n students S are sometimes distracted or show other interests. In such a case, the movement of the line of sight of a small number of students will be different. The student-side information of each of the small number of students S who showed eye-gaze movement can also be excluded from the calculation of eye-gaze alignment.
The information on the variation in the line of sight of the n students S is information that cannot be obtained from the instructor's side information, and is information that serves as "weighting" for adding information on the context of the explanation at that timing. For this reason, after creating a new on-demand lecture content using the lecturer's line of sight Lt and providing it to n students S to record the line of sight, the information to be this "weighting" is generated, and added as an afterthought. It can also be provided to a new student S as optional information. It can also be provided to the lecturer T as reference information for future lectures.

In addition, the quality of a lecture is greatly affected not only by the manner in which the lecture is conducted, such as the speaking style of the lecturer T and the method of explanation, but also by materials used in the lecture (eg, resume R, etc.). Specifically, for example, even if the content of Lecturer T's lecture is appropriate, the order in which the materials used in the lecture and the explanations are given may be different, or explanations may be given that are not related to the materials. The amount of information in the explanation may be excessive. In such a case, since there is a discrepancy between the material and the lecturer T's explanation, the student S cannot properly judge where to look in the material, and the contents of the lecture cannot be understood. difficult to understand. If there is a discrepancy between the materials and the lecturer T's explanation, the line of sight of the students S will vary, it will be difficult for the individual students S to synchronize their lines of sight, and the distribution of the line of sight of the students S as a group will also vary. growing.
On the other hand, even if lecturer T's explanation includes content that is not included in the material, if lecturer T can verbally teach the main points in line with the content of the lecture, student S will be able to use the material appropriately. It is possible to understand the contents by following with the eyes. In this case, the probability of occurrence of line-of-sight synchronization increases.
Thus, in this service, it is possible to determine whether or not the material used by the lecturer T matches the actual explanation of the lecturer T by using the occurrence rate of line-of-sight synchronization and the time distribution.

Traditionally, the eyes are said to be the window to the soul. Therefore, the degree of understanding of the student S can be estimated based on the student's line of sight Ls with respect to the lecturer T's explanation. In the on-demand lecture, not only are the lecturer T and the student S looking at the same position as line-of-sight synchronization, but (1) the line of sight of the lecturer T moves first and the line of sight of the student S follows, and (2) the students. The line of sight that expresses the understanding of the student S for the lecture, such as S pre-reading the resume R by predicting the future of the lecturer T's explanation, (3) the student S returning to the previous explanation and confirming the content. movement is expected. In this service, the quality of learning is evaluated by detecting the feature amount of the interaction between the instructor's line of sight Lt and the student's line of sight Ls, and estimating the internal state of each student S. .

FIG. 2 is a diagram showing a specific example of a screen displayed on a student terminal during a lecture.

When the lecturer T gives a lecture while looking at the resume R displayed on at least a part of the screen of the lecturer terminal 2, the contents of the lecture and the lecturer's line of sight Lt are recorded at the same time, and the content of the on-demand lecture is created. .
When the student S attends the on-demand lecture, the student S operates the student terminal 3 to display the resume R and the captured image of the instructor T's face on at least a part of the screen. When a student S attends an on-demand lecture, the face of the student S is imaged and the student's line of sight Ls is recorded during the lecture. Then, the instructor side information and the student side information indicating which part of the resume R each of the instructor T and the student S is viewing are obtained. Then, line-of-sight synchronization is detected based on the instructor side information and the student side information. AI (artificial intelligence) technology is used to detect line-of-sight alignment. In addition, there is no particular limitation on how the calibration process is performed to align the instructor's line of sight Lt and the student's line of sight Ls with the position on the resume R screen. Used. That is, first, predetermined marks are displayed in order at the four corners of the resume R, and the student S gazes at the marks. Then, at the moment when the character replacing the mark is displayed, the fact is verbally reported.

FIG. 3 is a graph showing the line-of-sight distance between the student and the lecturer.

The vertical axis of the graph shown in FIG. 3 indicates the line-of-sight distance when the horizontal width of the resume R is set to 1 as the reference length. Also, the horizontal axis indicates the number of frames in 208 seconds (208 seconds×25 fps=5200 frames). Specifically, the line-of-sight distance between the student's line of sight Ls and the lecturer's line of sight Lt for 208 seconds for viewing one resume R is shown. The line-of-sight distance can be used as information for evaluating the quality of student S' participation in class.
As shown in FIG. 3, there are a section including many eye-to-sight distances of 0.2 or less (the section surrounded by the dashed square frame) and a section including many eye-to-sight distances of 0.6 or more (the solid-line rectangle frame). two sections) are characteristic. Since the inter-line-of-sight distance is small in a section in which line-of-sight alignment occurs, there is a high probability that line-of-sight alignment has occurred in a section including many inter-line-of-sight distances of 0.2 or less. Note that even in a section including many eye-to-sight distances of 0.2 or less, there is a timing at which a large eye-to-sight distance suddenly appears. There is a high possibility that this is the timing when the line of sight of the lecturer T or the student S moves rapidly due to a saccade or the like, which is a rapid eyeball movement for capturing the visual target at the fovea centralis of the retina, which has high visual acuity. Sections that include many distances between the line of sight of 0.6 or more indicate that the line of sight Lt of the lecturer and the line of sight of the student Ls are separated by 0.6 times the width of the resume R, that is, at least to the extent that the resume R is divided into left and right. ing. However, the duration of the section containing a lot of eye-to-sight distances of 0.6 or more is short. It may last for seconds.
Hereinafter, such a sudden or persistently large inter-line-of-sight distance will be described with reference to FIG.

FIG. 4 is a graph showing the line-of-sight distance between another student and the lecturer different from FIG.
That is, the graph shown in FIG. 4 shows the distance between the gazes of the teacher T and another student S different from the student S in the example of FIG.

The vertical and horizontal axes in the graph shown in FIG. 4 are the same as in FIG. As shown in FIG. 4 , in the section surrounded by the solid-line square frame, a long eye-to-sight distance of 0.5 or more intermittently occurs. In addition, the occurrence of other long eye-to-sight distances shows a pattern different from that of the student S shown in FIG. In the section framed by the solid-line square, the distance between the line of sight was long because the student S was looking at the notebook at hand while looking away from the student terminal 3 . In addition, since the student S appropriately follows the lecturer T's explanation in the section where the eye-to-sight distance is short, as in FIG.

As shown in FIG. 4 and FIG. 3 described above, when eye-gaze synchronization occurs in a section where the eye-to-eye distance of the student S is short, (1) there is a small sudden increase in the eye-to-eye distance, but the eye-to-eye distance is generally A high proportion of them are 0.2 or less. Also, in the section where the line-of-sight distance of 0.5 or more occurs, (2) when the line-of-sight distance suddenly increases for a short period of time, and (3) when the line-of-sight distance increases for a certain time width. , (4) intermittently for a long period of time, the distance between the eyes continues to be long. Behind these (1) to (4) are the instructor's line of sight Lt and the student's line of sight Ls, which are the grounds for their occurrence, and the internal state of the student S that causes them.
Hereinafter, the relationship between the instructor's line of sight Lt, the student's line of sight Ls, and the distance between the lines of sight will be described with reference to FIG.

FIG. 5 is a graph showing the relationship between the instructor's line-of-sight movement speed, the student's line-of-sight movement speed, and the distance between the lines of sight, corresponding to the example of FIG.

In the upper, middle, and lower graphs of FIG. 5, the line-of-sight distance, the student's line-of-sight movement speed, and the lecturer's line-of-sight movement speed are shown, respectively.
The “movement speed of the line of sight” refers to the distance of movement of the line of sight between consecutive frames. Also, in the graph of FIG. 5, a plurality of sections are indicated by a plurality of types of vertically extending dashed lines. Of these lines, broken lines Lf1 to Lf7 indicate locations where the moving speed of the line of sight of the instructor T is faster than before and after. Broken lines Li1 to Li7 indicate locations where the line of sight of the student S impulsively (impulsively) moves.

As described above, in this service, when the line-of-sight distance between one lecturer T and n students S is short, there is a high probability that line-of-sight synchronization has occurred. However, based only on the fact that the line-of-sight distance is short, it can be estimated that lecturer T and student S are looking at positions close to resume R, and that the lecture is likely to be focused. information cannot be obtained. On the other hand, since the line of sight is said to be the window of the mind as described above, it can be expected that the detailed interaction of the line of sight between the lecturer T and the student S expresses more than that. Line-of-sight synchronization is a phenomenon in which the line of sight of student S follows the line of sight of lecturer T, and there are several interaction patterns in the change in the distance between the lines of sight of the two. In this service, these patterns are evaluated, and the evaluation results are used as useful information for estimating the student's internal state. A specific example of the pattern of change in the distance between eyes will be described below with reference to the graph of FIG.

First, the patterns of change in the distance between the sight lines are classified into five types of patterns A to E. FIG. Of these patterns, pattern A is a change pattern in which the student's line of sight Ls follows the lecturer's line of sight Lt. In the case of pattern A, the lecturer T first moves his line of sight to a new position and explains what is written there, and the student S notices this and makes his line of sight follow. In the pattern A, the line of sight of the lecturer T moves first to increase the line-of-sight distance, and then the line of sight of the student S follows the line of sight to shorten the line-of-sight distance.

Pattern B is a change pattern in which the student's line of sight Ls returns to confirmation. In the case of pattern B, the student S returns his line of sight to the position previously explained by the lecturer T to confirm his recognition, and then moves his line of sight to follow the lecturer T's explanation. It is presumed that such a movement of the line of sight indicates that the student S feels uneasy about understanding the lecture and tries to review the material immediately before. In pattern B, the line of sight of student S moves back to the previously seen position, thereby increasing the distance between the viewpoints, and after a short interval, the line of sight of student S follows the line of sight of lecturer T again. By moving like this, the distance between viewpoints is shortened.

Pattern C is a change pattern in which the student's line of sight Ls looks ahead. In the case of pattern C, the student S looks at the position of the next explanation so as to read ahead the lecturer T's explanation, and the lecturer T's explanation follows it. It is presumed that such a movement of the line of sight expresses the intention of the student S to actively try to understand the lecture. In pattern C, the line of sight of student S moves greatly toward a position that has not been seen before, thereby increasing the distance between the viewpoints. the distance between them is reduced.

Pattern D is a change pattern in which the instructor T looks away. In the case of pattern D, the line of sight of the lecturer T leaves the area viewed by the student S, and returns again after a short interval. For example, when the lecturer T sees something other than the resume R, the lecturer's line of sight Lt moves outside the range of the resume R. The line-of-sight distance at this timing is not used for estimating the student's S internal state.

Pattern E is a change pattern in which student S looks away. In the case of pattern E, the line of sight of the student S moves outside the range of the resume R. In pattern E, since the behavior of the student S taking notes or looking at the notes includes the action of looking away from the student terminal 3, a more detailed analysis is performed.

Here, with reference to FIG. 5, an example will be described in which the moving speed of the line of sight between the two is added to the distance between the lines of sight, and the movement of the line of sight, which is the background to the change in the distance between the lines of sight, is analyzed and evaluated in more detail.
The graphs in FIG. 5 show the distance between the line of sight Lt of the instructor and the line of sight Ls of the student (upper graph), the speed of the instructor's line of sight Lt (lower graph), and the speed of the student's line of sight Ls (lower graph). middle graph) are shown along the time axis.

The graph of FIG. 5 represents a plurality of sections each indicated by any two of the above-described broken lines Lf1 to Lf7 and broken lines Li1 to LI7. Specifically, a section a1 indicated by dashed lines Li1 and Li2, a section a2 indicated by dashed lines Lf1 and Li3, a section a3 indicated by dashed lines Li4 and Li5, a section a4 indicated by dashed lines Lf2 and Lf3, and a A section a5 indicated by Lf4 and Li6, a section a6 indicated by broken lines Lf5 and Lf6, and a section a7 indicated by broken lines Lf7 and Li7 are shown.

Among these multiple sections, the section a2 denoted by A1, the section a5 denoted by A2, and the section a7 denoted by A3 are the distances between the line of sight indicating the features corresponding to the pattern A described above. It's a big section.
In sections a2, a5, and a7, the instructor's line of sight Lt continuously and greatly moves at the timing when the distance between the lines of sight increases. Further, at the timing when the inter-viewpoint distance decreases, the student's line of sight Ls moves impulsively. This is because Lecturer T's line of sight movement is not saccade-like, but as the line of sight moves gradually according to the progress of the explanation of the resume R, the distance between the line of sight widens, and when it accumulates to some extent, the distance between the line of sight widens. It is presumed that the student S notices and that his line of sight follows the content of the explanation and moves in a saccade-like manner.
Specifically, for example, in section a7, when lecturer T finished explaining the lower left part of resume R, he suddenly moved his line of sight to the upper right of resume R and shifted to the next explanation described there. It took about 2 seconds for the student S to notice the change, and then he moved his line of sight to follow the upper right of the resume R.

Further, the section a1 indicated as B1 and the section a3 indicated as B2 are sections having a long inter-line-of-sight distance indicating the feature corresponding to the pattern B described above.
In the sections a1 and a3, the line-of-sight movement of the student S occurs impulsively at the start and end of the section with a long line-of-sight distance. On the other hand, the line of sight of the lecturer T hardly moves. This is because the student's line of sight Ls once leaves the lecturer's line of sight Lt, and then returns again. Although pattern B and pattern E cannot be distinguished only by this, pattern B is characterized by a short duration of the section.
Specifically, for example, in the section a1, the student S listens to the lecturer T and suddenly moves his line of sight to confirm the title of the resume R, and then returns his line of sight to the position corresponding to the lecturer T's explanation. rice field. As a result, an impulse-like distance change occurred in the student's line of sight Ls.

Also, the section a4 denoted by D1 and the section a6 denoted by D2 are sections having a long inter-line-of-sight distance, which indicates the feature corresponding to the pattern D described above.
In sections a4 and a6, the instructor's line of sight Lt moves significantly at the start and end of the section with a long distance between the lines of sight, and then returns again within a short period of time. However, the time width is short, and it is not possible to judge what the lecturer T is watching in each of the sections a4 and a6 from the graph of FIG.

In addition, in the example of FIG. 5, the line-of-sight movement corresponding to the pattern C described above is not found. As a tendency, changes in the line-of-sight distance are often related to relatively slow and continuous changes in the instructor's line of sight Lt and saccade-like rapid changes in the student's line of sight Ls. One of the reasons for this is the student's passive follow-up. In other words, the lecturer T moves his/her line of sight as he systematically explains, but the lecturer T's explanation progresses ahead of the lecturer T when the student S is looking at a position of the resume R. When it notices that it's gone, it rushes after it. The student S has such passive follow-up. The presence or absence of passive follow-up can be specified by performing a relative analysis including student-side information of other students S who attend the same on-demand lecture.

A case has been described above in which the student S seriously listens to the lecturer T's explanation and looks at the resume R, but there may be some students S who do not listen to the explanation seriously. The timing at which the student S is not looking at the resume R can be easily detected from the angle of the line of sight. On the other hand, in order to estimate that the student S is looking at another position of the resume R without listening to the lecturer T seriously, analysis is required. Therefore, a case where the student S does not take the on-demand lecture seriously will be described with reference to FIGS. 6 to 8. FIG.
(A) of FIG. 6 is a graph showing the change over time of the line-of-sight distance when line-of-sight synchronization occurs.
(B) of FIG. 6 is a graph showing the change over time of the distance between eyes in a scene where the student looks at the resume but the eyes are out of alignment. Specifically, the graph of (B) in FIG. 6 is obtained by exchanging the first half section and the second half section of the line of sight of the student in the graph of (A) in FIG. It is a graph synthesized when no tuning occurs.
FIG. 7 is a histogram showing the distribution of the line-of-sight distance between the graph of FIG. 6(A) and the graph of FIG. 6(B).
FIGS. 8A and 8B are graphs showing the distribution of the line-of-sight distance in other resumes that were not used in the analysis.

As shown in the graph of FIG. 6(B), there are some timings when the instructor's line of sight Lt and the student's line of sight Ls accidentally approach each other and it seems as if line-of-sight synchronization has occurred. A state in which the line-of-sight distance is widened and line-of-sight synchronization is not occurring (such a state is hereinafter referred to as “line-of-sight non-synchronization”). To make the difference clearer, a histogram of the line-of-sight distance is shown in FIG.

As shown in FIG. 7, the distribution when line-of-sight alignment occurs is significantly different from the distribution when line-of-sight non-synchronization occurs. A curve Lh1 in FIG. 7 indicates the distribution of the distance between the lines of sight when line-of-sight synchronization occurs. A curve Lh2 indicates the distribution of the distance between the lines of sight when line-of-sight non-synchronization occurs. When line-of-sight alignment occurs, most of the line-of-sight distances are 0.2 or less. Located in section 4. By taking the histogram shown in FIG. 7 as a probability distribution, the two waveforms shown in FIG. 7 can be easily identified. In order to confirm the generality of the distribution of the distance between the gazes when gaze synchronization occurs, two resumes R different from the resumes R corresponding to the explanations of FIGS. Graphs showing the distribution of the line-of-sight distance are shown in FIGS. 8A and 8B.

The steps of the horizontal axis of the graphs shown in FIGS. 8A and 8B are 0.05 units. In addition, all of the four students S who attended the lecture using the two resumes R had the same distribution of the distance between the gazes in FIG. shown in the graph. That is, in both the case of FIG. 7 and the cases of (A) and (B) of FIG. 8, it is assumed that the student S is requested in advance to seriously watch the lecture. Therefore, the data shown in FIG. 7 and FIGS. 8A and 8B can all be based on the assumption that line-of-sight alignment has occurred. Looking at FIG. 7 and FIGS. 8A and 8B under this premise, the distribution of the distance between gazes when eye-gaze synchronization occurs is shown in the resume R used in the lecture. They are similar regardless. Therefore, it can be said that the distribution pattern at the time of line-of-sight synchronization in FIG. 7 is general. In addition, this distribution pattern can be viewed as it is as a probability distribution of the distance between eyes by normalization. As a result, the internal state of the student S, including whether or not he or she is taking the lecture seriously, is determined by calculating the likelihood when the student sight line Ls of the new student S is obtained. It can be used as an indicator of

(A) of FIG. 9 is a diagram in which the line of sight of n students is output to the median filter.
FIG. 9B is a graph showing the instructor's line of sight.
(C) of FIG. 9 is a graph showing the line-of-sight distance between the n students and the lecturer.

By filtering the student's line of sight Ls of each of the n students S at the same timing, it is possible to represent one piece of line-of-sight trajectory data (hereinafter referred to as "line-of-sight trajectory data"). As a result, the line-of-sight trajectory data can be used as reference data instead of the line of sight of the lecturer T for the student S who attends the lecture while looking at the same material (eg, resume R). A filtering method is not particularly limited, and for example, an average value can be used. However, the average value is affected by even one exceptional value. Therefore, in this embodiment, a median filter that is less susceptible to the influence of exceptional values is used. The median filter sorts the X-coordinate and Y-coordinate of the line of sight of the n students S at each timing, and takes the central value (for example, the average of the two central values in the case of an even number). As a result, the influence of the maximum and minimum values, which tend to be exceptional values, can be eliminated.

Specifically, for example, when n=4 (the number of students S is four), the average value of the sorted second and third values is output. FIG. 9A shows the line-of-sight trajectory data of the student S output as described above. Comparing the line-of-sight trajectory data shown in FIG. 9A and the lecturer line-of-sight Lt shown in FIG. orbits are decreasing.

(C) of FIG. 9 shows the course of the inter-line-of-sight distance between the line-of-sight trajectory data shown in (A) of FIG. 9 and the instructor line of sight Lt shown in (B) of FIG. The line-of-sight distance pattern shown in FIG. 9C is similar to the line-of-sight distance pattern shown in FIG. However, the large inter-viewpoint distance and impulse in the vicinity of frames 0 to 400 seen in FIG. 6A are not seen in the inter-sight distance pattern shown in FIG. 9C. In other words, it can be seen that this portion was the inter-line-of-sight distance that occurred because the movement of the student's line of sight Ls did not follow the movement of the instructor's line of sight Lt. In this way, the line-of-sight trajectory data based on the student's line-of-sight Ls of the n students S can reduce the influence of fluctuations in the student's line-of-sight Ls of each student S, so that a more accurate line-of-sight distance can be obtained. can be detected.

In summary, in this service, the line-of-sight distance and the line-of-sight movement speed between the instructor and the student are detected based on the instructor side information and the student side information. Since the degree of line-of-sight synchronization can be estimated from the line-of-sight distance, it is possible to more precisely estimate the internal state of the student S based on the line-of-sight movement speed.

In addition, in this service, scenes in which the distance between eyes is wide are analyzed. As a result of the analysis, a pattern is estimated, for example, that the student S looks at or writes in a notebook, that the student S goes back to the previous explanation part, or that the student S reads ahead to predict the next. These estimations are directly linked to the estimation of the learning quality of the student S. In addition, the reason why the line-of-sight distance is widened is on both the instructor T side and the student S side, and it is analyzed what kind of meaning it has in terms of learning interaction. A skilled lecturer T observes the student S in a face-to-face class and infers this, but the line-of-sight synchronization method detected by this service also leads to such understanding of the student S's learning.

In addition, the gaze synchronization detection method adopted in this service is effective not only for online lectures, including on-demand lectures, but also for face-to-face lectures. In addition, this service can also be applied to lectures in a hybrid format of online lectures and face-to-face lectures. For example, it can be applied when students S gather in a classroom and each student S takes a lecture while watching the screen of the student terminal 3 . At that time, the lecturer T must individually grasp the learning situation of all the students S in such an environment. In other words, in a conventional lecture using a paper-medium textbook, all students S have the same textbook at hand, so it is rather easy to grasp their participation in the class. If this is replaced by a student terminal 3 such as a personal computer or a tablet, even if the student S is looking at the screen of the student terminal 3, it will not be possible to judge the internal state such as the intention of participation and the degree of understanding from the side. becomes extremely difficult. In order to judge this internal state, it is necessary for the student S, who is actually watching, to confirm the contents of the screen of the student terminal 3. , it is practically impossible to teach while grasping the internal situations of all students S.

On the other hand, according to the present service, it is possible to easily identify a student S who has had line-of-sight non-synchronization for a relatively long time in the lecture. As a result, the lecturer T can mainly follow the student S, so that the above problem can be solved.

Furthermore, recently, in a space as large as a classroom, a technique for extracting the line of sight of each person present there has been realized. By using this technology, it is easy to evaluate the student's line of sight Ls following the lecturer's line of sight Lt even in a face-to-face lecture, so an effect equivalent to that of line-of-sight synchronization can be achieved.

Furthermore, this service can be applied not only to educational institutions, but also to all kinds of situations (for example, corporate training, meetings, presentations, etc.) where video display devices such as personal computers are used in general. Specifically, for example, in in-house training, it is essential to grasp the state of understanding of the training content of the trainee S in order to ensure the effectiveness of the training. When this service is applied to a meeting or a presentation, it is possible to evaluate how much the audience corresponding to the above-mentioned student S understands and comprehends the contents, so how meaningful the place is. It is possible to evaluate whether there was

Next, with reference to FIG. 10, the configuration of an information processing system that realizes the provision of the above-described service, that is, an information processing system to which the server 1 according to one embodiment of the information processing apparatus of the present invention is applied will be described. do.
FIG. 10 is a diagram showing an example of the configuration of an information processing system to which a server according to an embodiment of the information processing apparatus of the invention is applied.

The information processing system shown in FIG. 10 is configured to include a server 1, a lecturer terminal 2, and student terminals 3-1 to 3-n.
The server 1, instructor terminal 2, and student terminals 3-1 to 3-n are interconnected via a predetermined network NW such as the Internet.

The server 1 is an information processing device managed by a service provider. The server 1 executes various processes for realizing this service while appropriately communicating with the instructor terminal 2 and the student terminals 3-1 to 3-n. Hereinafter, when there is no need to distinguish between the student terminals 3-1 to 3-n, they will be collectively referred to as "student terminals 3".

The lecturer terminal 2 is an information processing device operated by the lecturer T. FIG. The lecturer terminal 2 is composed of a personal computer, a tablet, a smartphone, etc., as described above. The lecturer terminal 2 receives various information input operations by the lecturer T and transmits the information to the server 1, and receives and displays various information (eg, resume R) transmitted from the server 1. FIG.

The student terminal 3 is an information processing device operated by the student S. FIG. The student terminal 3 is configured by a personal computer, a tablet, a smart phone, etc., as described above. The student terminal 3 receives various information input operations by the student S and transmits the information to the server 1, and receives and displays various information (eg, resume R) transmitted from the server 1. FIG.

11 is a block diagram showing an example of a hardware configuration of a server in the information processing system shown in FIG. 10. FIG.

The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an input section 16, and an output section 17. , a storage unit 18 , a communication unit 19 and a drive 20 .

The CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded from the storage unit 18 to the RAM 13 .
The RAM 13 also stores data necessary for the CPU 11 to execute various processes.

The CPU 11 , ROM 12 and RAM 13 are interconnected via a bus 14 . An input/output interface 15 is also connected to this bus 14 . An input unit 16 , an output unit 17 , a storage unit 18 , a communication unit 19 and a drive 20 are connected to the input/output interface 15 .

The input unit 16 is composed of, for example, a keyboard or the like, and inputs various information.
The output unit 17 includes a display such as a liquid crystal display, a speaker, and the like, and outputs various information as images and sounds.
The storage unit 18 is composed of a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 19 communicates with other devices (for example, the instructor terminal 2 and the student terminal 3 in FIG. 10) via the network NW including the Internet.

A removable medium 40 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted in the drive 20 as appropriate. A program read from the removable medium 40 by the drive 20 is installed in the storage section 18 as necessary.
The removable medium 40 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18 .

Although not shown, the lecturer terminal 2 in FIG. 10 can also have a configuration basically similar to the hardware configuration shown in FIG. Therefore, description of the hardware configuration of the instructor terminal 2 is omitted.

Various types of processing including internal state estimation processing can be executed by cooperation of various types of hardware and various types of software that constitute the information processing system of FIG. 10 including the server 1 of FIG. As a result, the service provider can provide the above-described present service to the on-demand lecture provider.
"Internal state estimation processing" refers to processing for estimating the internal state of the student S based on the lecturer side information and the student side information.
A functional configuration for executing internal state estimation processing executed in the server 1 shown in FIG. 11 constituting the information processing system shown in FIG. 10 will be described below.

12 is a functional block diagram showing an example of a functional configuration for executing an internal state estimation process among the functional configurations of the server of FIG. 11 that constitutes the information processing system of FIG. 10. FIG.

As shown in FIG. 12, when the server 1 executes the internal state estimation process, the CPU 11 includes a lecturer side information acquisition section 101, a student side information acquisition section 102, a detection section 103, and an estimation section 104. works.

In one area of the storage unit 18 of the server 1, a lecturer DB 181 and a student DB 182 are provided. In the lecturer DB 181, information that can uniquely identify the lecturer T (for example, an ID) is associated with lecturer-side information and managed. In the student DB 182, student-side information is associated with information (for example, an ID) that can uniquely identify a student S, and is managed.

The lecturer-side information acquisition unit 101 obtains a correspondence relationship between the lecturer's line of sight Lt, which is the movement history of the line of sight of the lecturer T who gives a lecture while viewing the resume R displayed on at least a part of the screen of the lecturer terminal 2, and the resume R. The information shown is acquired as instructor-side information. The instructor-side information acquired by the instructor-side information acquisition unit 101 is stored and managed in the instructor DB 181 .

The student-side information acquisition unit 102 acquires a student's line of sight Ls, which is the history of the movement of the student's line of sight while watching the resume R displayed on at least a part of the screen of the student's terminal 3. and resume R is acquired as information on the student side.

The detection unit 103 detects feature amounts of the lecturer side information and the student side information.
Specifically, for example, the detection unit 103 detects line-of-sight synchronization indicating a situation in which the line of sight of the student S follows the line of sight of the lecturer T as detection of the feature amount of the instructor side information and the student side information. .
Further, for example, the detection unit 103 detects the distance between the line of sight of the lecturer T and the line of sight of the student S as the detection of the feature amount of the lecturer side information and the student side information.
Further, for example, the detection unit 103 detects the difference between the movement speed of the instructor T's line of sight and the movement speed of the student S's line of sight as detection of the feature amount of the lecturer side information and the student side information.
Further, for example, the detection unit 103 detects the feature amount of the lecturer-side information and the student-side information, and detects the lecturer T and the class attendance outside the area of the resume R displayed on the respective screens of the lecturer terminal 2 and the student terminal 3. The line of sight of each person S is detected.

The estimation unit 104 estimates the internal state of the student S based on the feature amounts of the instructor-side information and the student-side information detected by the detection unit 103 .

As described above, the information processing apparatus of FIG. 11 has the above-described functional configuration shown in FIG. 12, so that the on-demand lecture provider can efficiently grasp the internal state of the lecture attendee. can be made

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the range that can achieve the object of the present invention are included in the present invention. Consider.

For example, in the above-described embodiments, an example in which the present service is applied to on-demand lectures is shown, but the present service can also be applied to online live lectures and face-to-face lectures. In other words, this service can be applied to any case in which information is transmitted while the information transmitting side and the information receiving side are viewing common materials on different screens. When extracting information such as line of sight and posture in an online live lecture, the extracted information is transmitted to the server 1 in real time, and line-of-sight synchronization is detected. Then, the detection result is sent to the student terminal 3 and fed back to the student S.

Further, for example, in the above-described embodiment, the resume R is exemplified as the material shared by the lecturer T and the student S, but not only a still image like a resume but also a moving image can be targeted.

Also, for example, the calibration method employed in the above-described embodiments is merely an example. It is desirable to employ a technique that allows the calibration to be completed without the student S noticing it as much as possible.

Also, the configuration of the screen of the student terminal 3 and the position and size where the resume R is displayed in FIG. 2 are merely examples. Also, although not shown, the configuration of the screen of the student terminal 3 and the position and size of the resume R displayed are not particularly limited.

Also, the system configuration shown in FIG. 10 and the hardware configuration of the server 1 shown in FIG. 11 are merely examples for achieving the object of the present invention, and are not particularly limited.

Also, the functional block diagram shown in FIG. 12 is merely an example and is not particularly limited. That is, it is sufficient that the information processing system of FIG. 10 has a function capable of executing the above-described internal state estimation processing as a whole. It is not limited to the example of FIG.

Also, the locations of the functional blocks and the database are not limited to those shown in FIG. 12, and may be arbitrary.
In the example of FIG. 12, the internal state estimation processing is configured to be performed under the control of the CPU 11 of the server 1 that constitutes the information processing system of FIG. 10, but is not limited to this. For example, at least part of the functional blocks and database arranged on the server 1 side may be provided on the instructor terminal 2 side, the student terminal 3 side, or another information processing device (not shown).

Also, the series of processes described above can be executed by hardware or by software.
Also, one functional block may be composed of hardware alone, software alone, or a combination thereof.

When a series of processes is to be executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware.
Also, the computer may be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smart phone, or a personal computer.

A recording medium containing such a program not only consists of a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but is also preinstalled in the device main body and delivered to the user. It consists of a provided recording medium, etc.

In this specification, the steps of writing a program recorded on a recording medium are not necessarily processed chronologically according to the order, but may be executed in parallel or individually. It also includes the processing to be executed.

In summary, the information processing apparatus to which the present invention is applied only needs to have the following configuration, and can take various forms.
That is, the information processing apparatus to which the present invention is applied is
The first screen (for example, the screen of the instructor terminal 2 in FIG. 1) is displayed at a predetermined position (for example, at least a part of the screen of the instructor terminal 2) while viewing a predetermined image (for example, the resume R in FIG. 1). Acquire first information (for example, the above-described instructor-side information) indicating the correspondence relationship between the history of line-of-sight movement of one user (for example, the lecturer T in FIG. 1) (for example, the lecturer's line-of-sight Lt in FIG. 1) and the predetermined image. a first acquisition means (instructor-side information acquisition unit 101 in FIG. 12);
A second user receiving the information while viewing the predetermined image displayed at a predetermined position (for example, at least a part of the screen of the student terminal 3) on the second screen (for example, the screen of the student terminal 3 in FIG. 1). Acquire second information (for example, the above-mentioned student-side information) that indicates the correspondence relationship between the movement history of the line of sight (for example, the student S in FIG. 1) (for example, the student's line of sight Ls in FIG. 1) and the predetermined image. A second acquisition means (student side information acquisition unit 102 in FIG. 12) to
detection means (the detection unit 103 in FIG. 12) for detecting the feature amount of the first information and the second information;
estimating means (estimating unit 104 in FIG. 12) for estimating the internal state of the second user based on the detected feature quantity;
Information processing device.

That is, the first information indicating the correspondence relationship between the history of the movement of the line of sight of the first user who transmits the information while viewing the predetermined image on the first screen and the predetermined image, and the transmission of the information while viewing the predetermined image on the second screen. Second information indicating the correspondence relationship between the history of the movement of the line of sight of the second user who receives the image and the predetermined image is acquired. Then, the feature amount of the acquired information is detected, and the internal state of the second user is estimated based on the feature amount.
This makes it possible to grasp the internal state of the second user from the detected feature quantity regardless of the timing at which the first information and the second information are acquired.

Further, the detection means
As the detection of the feature amount, it is possible to detect a situation in which the line of sight of the second user follows the line of sight of the first user (for example, the above-described line-of-sight synchronization).

In other words, a situation in which the line of sight of the second user follows the line of sight of the first user is detected as the detection of the feature amount of the first information and the second information.
This makes it possible to grasp the internal state of the second user based on the situation in which the line of sight of the second user follows the line of sight of the first user.

Further, the detection means
As the detection of the feature quantity, the distance between the line of sight of the first user and the line of sight of the second user can be detected.

That is, the inter-viewpoint distance between the line of sight of the first user and the line of sight of the second user is detected as the detection of the feature amount of the first information and the second information.
This makes it possible to grasp the internal state of the second user based on the inter-viewpoint distance between the line of sight of the first user and the line of sight of the second user.

Further, the detection means
As the detection of the feature amount, a difference between the moving speed of the line of sight of the first user and the moving speed of the line of sight of the second user can be detected.

That is, the difference between the moving speed of the line of sight of the first user and the moving speed of the line of sight of the second user is detected as the detection of the feature amount of the first information and the second information.
This makes it possible to grasp the internal state of the second user based on the difference between the movement speed of the line of sight of the first user and the movement speed of the line of sight of the second user.

Further, the detection means
As the detection of the feature amount, the line of sight of each of the first user and the second user outside the area of the predetermined image can be detected.

That is, as the detection of the feature amount of the first information and the second information, it is possible to detect the line of sight of each of the first user and the second user outside the area of the predetermined image.
As a result, it is possible to grasp the internal state of the second user based on the line of sight of the first user and the second user outside the area of the predetermined image.

1... server, 2... lecturer terminal, 3... student terminal, 11... CPU, 12... ROM, 13... RAM, 14... bus, 15... input Output interface 16 Input unit 17 Output unit 18 Storage unit 19 Communication unit 20 Drive 40 Removable media 101 Instructor side Information acquisition unit 102 Student side information acquisition unit 103 Detection unit 104 Estimation unit 181 Instructor DB 182 Student DB NW Network T... lecturer, S... student, R... resume, Lt... lecturer's line of sight, Ls... student's line of sight

Claims (6)

a first acquisition means for acquiring first information indicating a correspondence relationship between a history of movement of a line of sight of a first user who transmits information while viewing a predetermined image displayed at a predetermined position on a first screen and the predetermined image; ,
Acquiring second information indicating a correspondence relationship between a history of movement of the line of sight of a second user to whom the information is transmitted and the predetermined image while viewing the predetermined image displayed at a predetermined position on the second screen; 2 acquisition means;
detection means for detecting the feature amount of the first information and the second information;
estimating means for estimating the internal state of the second user based on the detected feature quantity;
Information processing device. The detection means is
Detecting a situation in which the line of sight of the second user follows the line of sight of the first user as the detection of the feature amount.
The information processing device according to claim 1 . The detection means is
Detecting the distance between the line of sight of the first user and the line of sight of the second user as the feature amount detection,
The information processing device according to claim 1 . The estimation means is
Detecting the difference between the movement speed of the line of sight of the first user and the movement speed of the line of sight of the second user as the detection of the feature amount,
The information processing device according to claim 1 . The estimation means is
Detecting the line of sight of each of the first user and the second user outside the area of the predetermined image as the detection of the feature amount,
The information processing device according to claim 1 . The computer that controls the information processing device,
a first acquisition step of acquiring first information indicating a correspondence relationship between a history of movement of a line of sight of a first user who transmits information while viewing a predetermined image displayed at a predetermined position on a first screen and the predetermined image; ,
Acquiring second information indicating a correspondence relationship between a history of movement of the line of sight of a second user to whom the information is transmitted and the predetermined image while viewing the predetermined image displayed at a predetermined position on the second screen; 2 an acquisition step;
a detection step of detecting a feature amount of the first information and the second information;
an estimation step of estimating the internal state of the second user based on the detected feature quantity;
A program that executes control processing including

Images