JP2001228785A - Method and device for transmitting computer support information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve information transmitting efficiency in an information transmitting system in which plural computer systems are connected together. SOLUTION: Viewpoint information of a transmitting person and a receiving person on their respective display screens is displayed on the respective display screens. Thus, aiming condition of visual material information is transmitted to the opposite side, the visual material referring position is displayed and the degree of concentration of the information receiving person is comprehended.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a computer-aided information transmitting method, a computer-aided information transmitting apparatus for transmitting information between a sender and a receiver via a computer system, and a recording medium on which a program for the information is recorded. About.

[0002]

2. Description of the Related Art There has been proposed an educational method and an educational system in which a teacher and a student face each other's display device and remotely give a lesson while displaying a lecture text on a screen of a display device in a computer system. Such an educational method and an educational system can be considered as an information transmitting method and an information transmitting system for transmitting various information.

[0003]

In the distance learning method using a computer system as described above, since the attitude (posture) of the teacher and the student is not transmitted, the distance learning method is used in a face-to-face education in a general classroom. Difficult education. This means that in general information transmission, the teacher corresponds to the sender and the student corresponds to the recipient.

One object of the present invention is to provide a computer-aided information transmitting method and a computer-aided information transmitting apparatus capable of improving the efficiency of information transmission from a sender to a receiver in information transmission via a computer system. Is to do.

Another object of the present invention is to propose a specific method and apparatus for improving information transmission efficiency.

[0006]

SUMMARY OF THE INVENTION According to the present invention, visual data information is transmitted from a transmitting computer system to a receiving computer system, and the visual data information is displayed on the display screens of the display devices of the transmitting and receiving computer systems. A computer-assisted information transmitting method for transmitting, to a receiving computer system, voice knowledge information that is described by voice while a sender of the transmitting computer system refers to visual material information displayed on the display screen, wherein the display screen of the transmitting computer system is used. Detects the line-of-sight direction of the receiver with respect to the display screen of the sender and the receiver computer system, acquires the line-of-sight information of both, and displays the correlation state of the two line-of-sight information on the display screen of the transmitter computer system. It is characterized by the following.

Further, according to the present invention, the transmitting computer system and the receiving computer system are connected via a communication line, and the visual material information is transmitted from the transmitting computer system to the receiving computer system so that the transmitting and receiving computer systems are connected. The visual information is displayed on the display screen of the display device, and the sender of the transmitting computer system transmits the audio knowledge information described by voice to the receiving computer system while referring to the visual information displayed on the display screen. In the computer-aided information transmission device, the transmitting-side computer system detects a line-of-sight direction of the sender with respect to the display screen to generate sender line-of-sight information, and the sender line-of-sight information The sender's line-of-sight information transmitting means for transmitting The receiving side computer system includes a sender's line of sight information receiving means for receiving the sender's line of sight information, and a sender's line of sight information displaying means for displaying the sender's line of sight information on a display screen of the receiving side computer system. And

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing an outline of a computer-aided information transmission system according to the present invention.

A transmitting computer system (device) 1 and a receiving computer system (device) 2 are connected via a network system 3 which is a communication line. The receiving-side computer system 2 can be arbitrarily connected from one to n.

Since the transmitting computer system 1 and the receiving computer system 2 have substantially the same structure, the hardware of the transmitting computer system 1 will be described in detail below.

The computer system 1 comprises a display device 4, a processing device 5, a transmitting / receiving device 6, three cameras 7 to
9, input device 10, microphone 11, speaker 1
2, the external storage device 13 and the like.

In front of the display device 4, a sender 1
00 should be seated on the chair and face to face. The three cameras 7, 8, and 9 are installed to accurately measure the line of sight of the sender 100. The camera 7 is a camera that captures an image from above the sender 100 and generates image data to capture changes in the planar position of the head of the sender 100 and changes in the rotation of the head. The camera 8 is a camera for generating image data for measuring the line of sight of the left eye of the sender 100. The camera 9 is a camera that generates image data for measuring the line of sight of the right eye of the sender 100. Here, three cameras 7 to 9 are used, but it is also possible to realize this with two or one camera by adding conditions to the attitude of the sender. Here, an example in which three cameras 7 to 9 are used for easy understanding of the principle will be described, but three cameras are not essential.

The input device 10 comprises a keyboard and a mouse.

FIG. 2 shows a detailed configuration of the hardware of the computer system 1. The processing device 5 is a CP
U14, main memory 15, image dedicated processor 16, image memory 17, bus 18, display interface (IF)
19, image IF20, input IF21, microphone I
F22, speaker IF23, external memory IF24, communication IF25, etc. Then, the CPU 14 captures image data from the cameras 7 to 9 via the image IF 20, captures input from the input device 10 via the input IF 21, captures audio data from the microphone 11 via the microphone IF, and The display data is output to the display device 4 via the display IF 19, and the audio data is output to the speaker 12 via the speaker IF 23. Also, visual material information such as lecture texts is fetched from the external storage device 10 via the external memory IF 24,
The receiving computer system 2 is connected to a network system (LAN) from the communication IF 25 via the transmitting / receiving device 6.
3 to communicate various kinds of information by communicating.

The processing device 5 analyzes the image data input from the cameras 7 to 9 to measure the plane coordinate position of the head of the sender 100, the rotation of the head, and the movement of the line of sight, and displays the display screen of the display device 4. Estimate the upper gaze point. This image data processing is performed at a high speed by using an image processor 16.
Use The input image data and the image data of the processing result are stored in the image memory 17. The processing program is stored in the main memory 15 and is configured to be transferred to the CPU 14 and executed at the time of execution.

FIG. 3 is a functional block diagram showing the configuration of software (processing program) of the transmission-side computer system 1. The other receiving computer systems 2 are similarly configured. The software 200 is stored in the main memory 15 and selectively activated and executed as needed.

The software 200 has a main function 2
10, gaze information generation function 220, and gaze information transmission function 23
0, gaze information reception function 240, and gaze information display function 260
Is provided.

The main function 210 includes an input function 211 for processing from a keyboard and a mouse, and a “transmission side” and a “reception side” for transmitting information to the computer system 1.
An initial setting connection function 212 for setting which function of the “transmission / reception side (bidirectional transmission)” to connect to any other computer system 2 and executing this setting, and visual material information used for information transmission From a database, a transmission visual material information setting function 213 that controls the progress of information transmission, a display function 215 that performs display control of the display device 4, and audio data input from the microphone 11. And a voice input transmission function 216 for transmitting the received voice data to the speaker 12.
And a communication function 218 for communicating with another computer system 2 via the network system 3.

The line-of-sight information generation function 220 includes the cameras 7 to 9
This is a function of measuring the line of sight of the sender (self) 100 by acquiring and analyzing image data from the device, and generating information information indicating the direction of the line of sight.

The line-of-sight information transmission function 230 is a function of transmitting the line-of-sight information of the sender 100 generated by the line-of-sight information generation function 220 to another computer system 2.

The line-of-sight information receiving function 240 is a function of receiving the line-of-sight information of the receiver (other party) generated by another computer system 2.

The line-of-sight information storage function 250 is a function of storing the line-of-sight information generated by its own computer system 1 and the line-of-sight information received from another computer system 2 in the main memory 15.

The gaze information display function 260 obtains a viewpoint position on the display screen of the display device 4 based on the gaze information generated by the own computer system 1, and displays a self viewpoint mark at that position.
An opponent viewpoint display function 262 for obtaining a viewpoint position on the display screen of the display device 4 based on the line-of-sight information received from the other computer system 2 and displaying an opponent viewpoint mark at the position; And a correlation state evaluation function 264 for calculating a correlation state (coincidence level = concentration degree) between the line-of-sight information generated in its own computer system 1 and the line-of-sight information received from another computer system 2. A correlation state evaluation result display function 265 for displaying a correlation state of the line-of-sight information from the visual material information display area in a list form, a display form selection function 266 for selecting a correlation state display form (display order or display partner), A subsequence that generates a small image that displays the trajectory of the mark and displays it separately from the visual information display area A display function 267 monitors the correlation state, and a warning function 268 correlation state is alert and worse than a predetermined state.

The present invention is a computer-aided information transmitting method and a computer-aided information transmitting apparatus for transmitting various kinds of information, and a recording medium on which a program for the computer-aided information transmitting apparatus is recorded. Will be described.

FIG. 4 is a block diagram of the functions of each of the computer systems 1 and 2 in a computer-aided education system configured by connecting such a plurality of computer systems 1 and 2 via a network system 3. is there.

Here, the computer system 1 constitutes a teacher computer system serving as a transmitting side, and the computer system 2 constitutes a student computer system serving as a receiving side. Student computer system 2
Is connected to an arbitrary number of computers. Here, the computer system 2 of the student A is exemplified.

When the teacher computer system 1 serving as the transmission side sets the teacher side (transmission side) in the initial setting connection function 212a, the main function 210a
Are the input function 211a, the initial setting connection function 212a, the transmission visual information setting function 213a, and the information transmission progress function 2
14a, display function 215a, and voice input transmission function 216a
And the communication function 218a, and the gaze information generation function 22
0a, gaze information transmission function 230a, and gaze information reception function 2
40a and the gaze information storage function 250a, and the gaze information display function 260a includes a self-viewpoint display function 261a, a partner viewpoint display function 262a, and a viewpoint movement locus display function 263.
a, a correlation state evaluation function 264a, a correlation state evaluation result display function 265a, a display mode selection function 266a, a sub scene display function 267a, and a processing program for functioning the alarm function 268a are started, and the set student (reception) side computer system is started. Connect to 2.

When the student computer system 2 as the receiving side sets the student side (receiving side) in the initial setting connection function 212b, the main function 2
10b is an input function 211b and an initial setting connection function 212
b, the information transmission progress function 214b, the display function 215b, the communication function 216b, and the voice reception output function 217b, and the gaze information generation function 220b and the gaze information transmission function 23
0b, gaze information receiving function 240b, and gaze information storage function 2
50b, the gaze information display function 260b activates a processing program that causes the self-viewpoint display function 261b, the opponent's viewpoint display function 262b, and the viewpoint movement locus display function 263b to function, and the set teacher (transmission) side computer system 1 Connect with

The setting and connection processing can be performed by a predetermined scheduler in addition to the joint operation of the computer systems 1 and 2 and the teacher and the student.

FIG. 5 is a flowchart of a processing program executed by the main function 210a in order for the computer system 1 to serve as a teacher (transmission) side computer system to construct such a computer-assisted education system and progress a lesson. .

First, the following processing is executed to realize the initialization connection function 212a.

Processing Step S212a-1 An initial screen for system construction is displayed and the input device 10
, The setting input of the “receiving (student) side computer system” to be connected to the function setting inputs of the “sending side (teacher)”, “receiving (student) side” and “transmitting / receiving side”

Processing Step S212a-2 Function setting processing and line connection processing are executed.

Processing Step S212a-3 The teacher-side initial screen is displayed.

Processing Step S212a-4 The student computer system 2 is instructed to display a student-side initial screen.

Next, the transmission visual material information setting function 213a
The following processing is executed in order to realize.

Processing Step S213a-1 On the teacher's initial screen, transmission visual material information (lecture text and other materials) used for the lesson is set based on the setting input from the input device 10.

Next, the following processing is executed to realize the information transmission progress function 214a.

Processing Step S214a-1 The teacher's computer system 1 displays the lesson teacher's display screen format on the display screen of the display device 4a in accordance with the instruction input from the input device 10, and the lesson student computer system 2 Instruct the display of the side display screen format.

Processing Step S214a-2 According to the instruction input from the input device 10, the transmission visual information to be used is taken out, displayed in the lesson teacher display screen format, and displayed in the lesson student display screen format. Send to This transmitted visual information
Update is performed according to the subsequent instruction input from the input device 10.

Then, the following processing is executed to realize the voice input transmission function 216a.

Processing Step S216a-1 An explanation (voice knowledge information) of the teacher's voice is acquired from the microphone 11 as voice data and transmitted.

FIG. 6 shows a main function 21 of the computer system 2 as a student (receiving) computer system.
0b is a flowchart of a processing program executed.

First, the following processing is executed to realize the initial setting connection function 212b.

Processing Step S212b-1 An initial screen for constructing the system is displayed, and the input device 10 is to be connected to the function setting inputs of "transmission (teacher) side", "reception (student) side" and "transmission / reception side". The setting input of "transmission (teacher) side computer system" is obtained.

Processing Step S212b-2 Function setting processing and line connection processing are executed.

Processing Step S212b-3 The student's initial screen is displayed based on the instruction from the teacher's computer system 1.

Next, the following processing is executed to realize the information transmission progress function 214b.

Processing Step S214b-1 The lesson student-side display screen format is displayed in accordance with the instruction from the teacher-side computer system 1.

Processing Step S214b-2 The transmitted visual material information transmitted from the teacher-side computer system 1 is received, and the transmitted visual material information is displayed in a class student display screen format. The transmitted visual material information to be displayed is updated in accordance with the update of the transmitted transmitted visual material information.

Then, the following processing is executed to realize the voice reception output function 217b.

Processing Step S217b-1 The sound data is received, converted into sound and supplied to the speaker 12 to generate sound.

In such a remote class, the line-of-sight information generating function 220a of the teacher computer system 1 obtains and analyzes the image data of the teacher's head from the cameras 7 to 9, thereby displaying the display screen of the display device 4a. Generates teacher's line-of-sight information for.

Next, the gaze information display function of the teacher (sender) and the student (receiver) will be described. This gaze information display function is a function for improving the efficiency of lessons (information transmission) by transmitting and displaying the gaze (viewpoint) of the teacher and the gaze (viewpoint) of the student on the display screen to the other party.

FIG. 7 is a flowchart of a process for realizing the line-of-sight information generation function 220a in the teacher-side computer system 1. This process is started periodically by a timer. Further, this processing program is executed by the gaze information generation function 220 in the student computer system 2.
It is also applied to realize b.

Processing Step S220a-1 Image data of the head of the teacher (student) is captured from the cameras 7 to 9.

Processing Step S220a-2 The position of the center of gravity of the head of the teacher (student) (distance X, Y from the origin of the screen coordinate system) is determined by analyzing the image data from the camera 7 above.

Processing Step S220a-3 The positions of both ears of the head are obtained from the image data from the camera 7 above, and the rotation angle Ω of the head is obtained from the positions of both ears and the position of the center of gravity of the head.

Processing Step S220a-4 The center of gravity of the head is determined based on the image data of the cameras 8 and 9 on both sides, and the three-point of the head is determined from the installation position of the cameras 8 and 9 and the geometric relationship between the two centers of gravity. The dimensional position (X, Y, Z) is obtained, and the image data of the cameras 8 and 9 is analyzed to obtain the inclination angle Φ of the head with respect to the vertical axis.

Processing Step S220a-5 Based on the image data of the cameras 8 and 9, the outline positions of both eyes are obtained, the position of the pupil is further obtained, and the angle β of the left and right deviation of the pupil is obtained based on these results. Obtain the angle γ.

Processing Step S220a-6 Three-dimensional position and posture information (X, Y, Z, Φ,
Ω) and the three-dimensional positions (x1, y1, z1), (x
2, y2, z2) and pupil orientation information (β1, γ1), (β
2, γ2), and the intersection (viewpoint) position data (u, u) of the line of sight and the display screen of the display device.
Find v).

FIG. 8 shows a detailed flowchart (a) of the viewpoint position determination processing in the processing step S220a-6 and an example of display screen division (b).

Processing Step S220a-6-1 The display screen is divided into a grid-like area and an area outside the screen divided into mxn equal parts, and area numbers are given to each of them. The area of each viewpoint position data (u, v) is calculated, the area number is added, and the area is stored in the table.

(B) shows an example in which the display screen is divided into 3 × 3. The grid area inside the screen is A1 to A9
And A0 outside the screen. The intersection (point of view) (u, v) between the teacher's line of sight and the display surface of the display device falls into one of the areas A0 to A9. It is important that the viewpoint (intersection) at each time of the teacher falls into any of A0 to A9. The viewpoint position data thus obtained is time-series teacher viewpoint position data as described below.

Time coordinate value Area number t0 (u0, v0) a1 t1 (u1, v1) a2 t2 (u2, v2) a3 ... tn (un, vn) an This time-series teacher viewpoint position The data is obtained by sampling and accumulating the viewpoint position data of the teacher at regular time intervals. In the case of a teacher, for example, if sampling is performed every 5 seconds, it can be easily understood that viewpoint position data as shown in FIG. 9 is obtained. Further, the viewpoint position data of the student is as shown in FIG. The data sampling times of the viewpoint position data of the teacher and the student are synchronized so that they are substantially the same time.

Then, the line-of-sight information transmitting function 230a and the line-of-sight information receiving function 240a execute a process of transmitting and receiving the line-of-sight information obtained as described above.

The gaze information storage function 250a stores the obtained gaze information (viewpoint position data) and the received gaze information (viewpoint position data) in the main memory (viewpoint table) 15 on the teacher side. Execute.

FIG. 11 shows the computer system 1 on the teacher side.
2 shows an example of the display content of the lesson teacher side screen of the display device 4 in FIG. The lecture text 401a, which is visual material information, is displayed in the visual material display area 402a. In the empty area 403a on the right side of the visual material display area 402a, a concentration level display field 404a for a plurality of students (recipients) is provided. By observing this, the sender can get an overview of the degree of concentration of the students. The sub-scene display area 4
05a, a viewpoint movement locus 406a indicating the movement state of the viewpoint of a specific student (for example, the viewpoint position data of the past minute sampled every 5 seconds and connected by a line) is selectively displayed. . Here, 407a is the current viewpoint mark of the student. 408a is the name of the student displaying the viewpoint movement locus 406a in the sub-scene display area 405a. 409a is a teacher's viewpoint mark. Although the viewpoint mark of the student can be displayed so as to be superimposed on the lecture text, it is not displayed here.

When the teacher observes the display screen and determines that the concentration of the specific student is lacking, the teacher uses the microphone 11 to specify the specific student (student name 408).
Voice alert can be issued to the computer system 2 of a) or to the computer system 2 of all students.

FIG. 12 shows the student computer system 2
5 shows an example of the display content of the lesson student-side display screen of the display device in FIG. Visual material display area 402b
In the figure, the received lecture text 401b is displayed in the same form as the transmitting side. 407b is a viewpoint mark of the student (recipient), 409b is a viewpoint mark of the teacher (sender), and 410b is a locus of movement of the viewpoint mark of the teacher. The viewpoint marks 407b and 409b can be switched so that the student's own viewpoint mark 407b or the teacher's viewpoint mark 409b can be displayed alone or both the student's and teacher's viewpoint marks 407b and 409b can be displayed. The distinction between the viewpoint marks 407b and 408b of the student and the teacher is made based on the shape and color of the mark.

Therefore, by observing such a display screen, the student can know the position of the visual material that the teacher is paying attention to and the moving direction thereof.

According to such a configuration, the teacher and the student
Although it is a remote class form, an environment similar to a face-to-face class form of a teacher and a student in a general classroom can be realized.

The screen display of such line-of-sight information is realized by the line-of-sight display function 260.

First, the gaze display function 260a in the teacher computer system 1 will be described.

FIG. 13 is a flowchart of a process for realizing the self-viewpoint display function 261a and displaying a teacher (sender) viewpoint mark.

Processing Step S261a-1 The current viewpoint position data of the teacher is acquired.

Processing Step S261a-2 The teacher's viewpoint mark 409 is superimposed on the lecture text 401a.
Display a.

FIG. 14 is a flowchart of a process for realizing the other party's viewpoint display function 262a and displaying the viewpoint mark of the student (recipient).

Processing Step S262a-1 Whether or not to display a viewpoint mark is set and input. The initial value (default) is set to “No” when set on the transmitting side, and set to “Necessary” when set on the receiving side.

Processing Step S262a-2 The processing branches according to the setting. If “No”, the viewpoint mark display processing is not performed.

Processing Step S262a-3 The current viewpoint position data of the student displayed in the sub-scene display area 405a is obtained.

Processing Step S262a-4 The student's viewpoint mark is displayed over the lecture text 401a. In the example shown in FIG. 11, the viewpoint mark display processing of the student is not performed.

FIG. 15 shows a view point movement locus display function 263a.
Is a flowchart of a process for displaying a trajectory movement locus of a student (recipient) by realizing the following.

Processing Step S263a-1 Whether or not to display a viewpoint locus is set and input. The initial value (default) is set to “No” when set on the transmitting side, and set to “Necessary” when set on the receiving side.

Processing Step S263a-2 The processing branches according to the setting. When “No” is set, the viewpoint locus display processing is not performed.

Processing Step S263a-3 Obtains the current and past viewpoint position data (one minute viewpoint position data sampled at 5-second intervals) of the student displayed in the sub-scene display area 405a.

Processing Step S263a-4 A trajectory connecting the viewpoint positions with a line is superimposed and displayed on the lecture text 401a. In the example shown in FIG. 11, the viewpoint locus display processing is not performed.

The degree of concentration of the student in the lesson can be estimated by the degree of correlation between the eyes of the teacher and the student. The correlation state evaluation function 264a calculates the degree of the correlation state of the line of sight (degree of coincidence = correlation value) based on the line of sight information of the teacher and the student, and functions to support the grasp of the degree of concentration of the student.

FIG. 16 is a flowchart of processing for realizing a correlation state evaluation function 264a for calculating and evaluating the correlation state (correlation value) of the teacher and student viewpoint position data. This processing is started at a fixed time interval. For example, it is activated once every minute or once every five minutes.

Processing Step S264a-1 The viewpoint position data within a predetermined time is fetched from the teacher's viewpoint position data table (see FIG. 9).

Processing Step S264a-2 The target student n for which the correlation state is evaluated is set to n = A.

Processing Step S264a-3 The viewpoint position data within a predetermined time is fetched from the viewpoint position data table (see FIG. 10) of the student n (= A). The start time and the sampling interval of the data of the teacher and the student should be matched to a practical level.

Processing Step S264a-4 A correlation state between the teacher and student viewpoint position data is calculated using the acquired viewpoint position data.

Processing Step S264a-5 It is confirmed whether or not the processing has been completed for all the students, and the processing branches.

Processing Step S264a-6 Student n for evaluating the correlation state is updated to n = n + 1, and the process returns to processing step S264a-3.

The calculation of the correlation state can be performed by various methods. For example, the calculation is performed as follows.

In order to facilitate the display of the calculation formula, the table shown in FIG.

F (i) = S Here, F: data value in FIG. 9 i: number (1 to n) from the top of the table S: data area number (0 to 9) n: number of data Regarding the student's viewpoint position data, the table of FIG. 10 is expressed by the following equation.

G (i) = S Here, G: data value in FIG. 10 i: number (1 to n) from the top of the table S: data area number (0 to 9) n: number of data FIG. 18 is a processing step S264 of calculating a correlation state (correlation value) indicating the degree of coincidence of the gaze of the teacher and the student.
It is a detailed flowchart of a-4.

An example of the correlation value calculation shown in FIG. 18 will be described.

Processing Step S264a-4-A The number of data is determined and is set to n.

Processing Step S264a-4-B Initialization processing required for calculation is performed.

Processing Step S264a-4-C The following processing is monitored and branched so as to be repeated by the number of data (1 to n).

Processing Step S264a-4-D The teacher's viewpoint F (i) is compared with the student's viewpoint G (i).

Processing Step S264a-4-E When the teacher's viewpoint F (i) matches the student's viewpoint G (i), 1 is added to SKN.

Processing step S264a-4-Fi is incremented by 1, and the process returns to processing step 264a-4-C.

Processing Step S264a-4-G A correlation value is calculated. The correlation value ξ1 is calculated by the following equation.

Correlation value ξ1 = (SKN / n) × 100 where SKN: Cumulative number of matched cases n: Number of data (number of sampling data) This correlation value が 1 corresponds to the gaze of teacher and student at the time of sampling. 100% when they completely match during the period
Becomes When they are half matched, the result is 50%. When the teacher's viewpoint concentrates on the boundary between the two areas, a case may occur where the degree of coincidence does not match the actual degree of coincidence.

Therefore, another example of the calculation of the correlation value will be described with reference to FIG.

Processing Step S264a-4-A The number of data is determined, and this is set to n.

Processing Step S264a-4-B Initialization processing required for calculation is performed.

Processing Step S264a-4-C The following processing is monitored and branched so as to be repeated by the number of data (1 to n).

Processing Step S264a-4-D The teacher's viewpoint F (i) is compared with the student's viewpoint G (i).

Processing Step S264a-4-E 'When the teacher's viewpoint F (i) matches the student's viewpoint G (i), 2 is added to SKN.

Processing Step S264a-4-H It is determined whether or not the area of the teacher's viewpoint F (i) and the student's viewpoint G (i) are adjacent to each other.

Processing Step S264a-4-J When the area of the teacher's viewpoint F (i) is adjacent to the area of the student's viewpoint G (i), 1 is added to SKN.

Processing step S264a-4-Fi is incremented by 1, and the process returns to step S264a-4-C.

Processing Step S264a-4-G 'A correlation value is calculated. This correlation value ξ2 is calculated by the following equation.

Correlation value ξ2 = (SKN / 2n) × 100 where SKN: cumulative number of matched cases n: number of data (number of sampled data) This correlation value ξ2 is determined by the viewpoint F (i) of the teacher and that of the student. Viewpoint G
Even when (i) is concentrated near the boundary of the area, the effect of relatively matching the actual situation can be obtained.

The correlation values ξ1,
The average value of (ξ2 + (ξ1 + ξ2) / 2) of ξ2 may be used as the correlation value of each student.

FIG. 19 is a flowchart of a process for realizing a correlation state evaluation result display function 265a for displaying a list of correlation state evaluation results (correlation values) of each student's eyes as a degree of concentration.

Processing Step S265a-1 Correlation value data indicating evaluation results of all students is obtained.

Processing Step S265a-2 The evaluation results of all students are arranged in the order of bad (small correlation value).

Processing Step S265a-3 It is determined whether the display is based on the selection table (described later) or the default.

Processing Step S265a-4 The correlation value (concentration level) of the designated student is preferentially displayed in the concentration level display field 404a according to the selection table.

Processing Step S265a-5 The correlation values (concentrations) of the students are displayed in the concentration display column 404a in the order determined by the default (in the order of the correlation values).

As described above, the concentration of the students can be displayed in a manner convenient for the teacher.

FIG. 20 is a flowchart of a process for realizing the display mode selection function 266a for selecting a student of the visual line information display.

Processing Step S266a-1 Input data from the input device 10 is monitored and a selection request input from a teacher is waited.

Processing Step S266a-2 It is determined whether or not there is a selection request.

Processing Step S266a-3 The contents requested by the teacher are fetched.

Processing Step S266a-4 The student selection table is updated.

FIG. 21 is a flowchart of a process for implementing the sub-scene display function 267a for displaying the trajectory of the student's line of sight on the display device 4 of the computer system 1 on the teacher side in detail. Also in this case, there are a display method using a selection table and a display method using a default, and the display is performed using any of the display methods.

Processing Step S267a-1 It is determined whether the display is based on the selection table or based on the default.

Processing Step S267a-2 Moving locus 4 of the student's viewpoint specified by the selection table
06a and the current viewpoint mark 407a are displayed in the sub-scene display area 405a.

Processing Step S267a-3 The viewpoint movement locus 406a of the student with the worst eye-gaze coincidence (correlation value) determined by default and the current viewpoint mark 407a are displayed in the sub-scene display area.

The warning function 268 blinks the warning sound generation and / or concentration level display field 404a and / or the sub-scene display area 405a for the student whose eye-gaze correlation value (degree of concentration) has fallen below a predetermined value. And inform the teacher.

Next, the gaze display function 260b of each student (receiving) in the computer system 2 will be described.

The self viewpoint display function 261b acquires the student's viewpoint position data and displays the viewpoint mark 407b in the same manner as the self viewpoint display function 261a in the teacher computer system 1 described with reference to FIG.

The partner viewpoint display function 262b has the initial value “necessary” and displays the teacher's viewpoint mark 409b as described with reference to FIG.

The viewpoint movement trajectory display function 263b has an initial value of "necessary" and displays the teacher's viewpoint movement trajectory 410b as described with reference to FIG.

According to such a computer-assisted education system, the teacher can grasp the student's class attitude and improve the lecture method (adjustment of explanation speed, presentation of reference visual data, warning, etc.) In addition, since the student can take a lesson in which the visual material reference efficiency is enhanced with reference to the teacher's attention position with respect to the visual material, an efficient lesson can be performed.

This computer-assisted education system is for reporting an answer or a proposal from a student by setting one student on the transmitting (teacher) side and setting the teacher and another student on the receiving (student) side. Can be used for Also,
By setting all the computer systems on the transmitting and receiving sides, it is also possible to configure so that each computer system transmits and receives visual information and voice and discusses.

In such a case, the correlation state evaluation function 2
64, correlation state evaluation result display function 265, display form selection function 266, sub-scene display function 267, alarm function 26
8 preferably functions only in the computer system 1 used by the teacher.

[0146]

According to the present invention, the following effects can be obtained.

1) Since the sender (teacher) side computer system can know the degree of concentration of the receiver (student), the sender can use the information transmission method (adjustment of explanation speed and presentation of reference visual data) And warnings).

2) The receiver (student) can display the sender's (teacher) 's viewpoint mark and viewpoint movement trajectory on the display screen of the display device in the receiver (student)' s computer system. By referring to the visual material (such as a lecture text) with reference to the teacher's viewpoint mark, the visual material reference efficiency is improved and the understanding of the information transmission content can be promoted.

[Brief description of the drawings]

FIG. 1 is a schematic diagram of a computer-aided information transmission system according to the present invention.

FIG. 2 is a block diagram showing a detailed configuration of hardware of a computer system according to the present invention.

FIG. 3 is a functional block diagram of a computer system according to the present invention.

FIG. 4 is a functional block diagram of the computer-aided education system according to the present invention.

FIG. 5 is a processing flowchart for realizing a main function executed by a teacher-side computer system in the computer-aided education system shown in FIG. 4;

FIG. 6 is a processing flowchart for realizing a main function executed by a student-side computer system in the computer-aided education system shown in FIG. 4;

FIG. 7 is a processing flowchart for realizing a line-of-sight information generation function 220a in the teacher-side computer system 1.

FIG. 8 is a detailed processing flowchart of a viewpoint position determination process and an example of display screen division.

FIG. 9 shows an example of teacher's viewpoint position data.

FIG. 10 shows an example of student's viewpoint position data.

FIG. 11 shows a display example of a lesson teacher side display screen.

FIG. 12 shows a display example of a lesson student-side display screen.

FIG. 13 is a processing flowchart for realizing a self-viewpoint display function.

FIG. 14 is a processing flowchart for realizing a partner viewpoint display function.

FIG. 15 is a processing flowchart for realizing a viewpoint movement locus display function.

FIG. 16 is a processing flowchart for realizing a correlation state evaluation function.

FIG. 17 is a process flowchart illustrating an example of a correlation value calculation process.

FIG. 18 is a processing flowchart illustrating another example of the correlation value meter processing.

FIG. 19 is a processing flowchart for realizing a function of displaying a result of evaluating the correlation between the eyes of a teacher and a student.

FIG. 20 is a processing flowchart for realizing a display mode selection function.

FIG. 21 is a processing flowchart for realizing a sub-scene display function.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Teacher computer system (device), 2 ... Student computer system (device), 3 ... Network system, 4 ... Display device, 5 ... Processing device, 6 ... Transmitting / receiving device, 7-9 ... Camera, 10 ... Input device , 11 microphone, 12 speaker, 13 external storage device, 1
4 CPU, 15 main memory, 16 image dedicated processor, 17 image memory.

Continuing from the front page (72) Inventor Hirokazu Sawano 5-2-1 Omikacho, Hitachi City, Ibaraki Prefecture Inside Hitachi Process Computer Engineering Co., Ltd. (72) Inventor Yoichi Takagi 5-2-1 Omikacho Hitachi City, Ibaraki Prefecture F-term (reference) 2C028 AA12 BA01 BA02 BA03 BA05 BB04 BB05 BB06 BC05 BD03 CA12 DA04 DA06 5B049 AA00 BB21 DD00 DD03 FF03 FF06 5B087 AE00 BC05 BC13 BC26 BC32 DG01 5K030 GA02 HA04 HBT01 CC02 CC08 DD13 JJ74 KK09

Claims (6)

[Claims] 1. A method for transmitting visual material information from a transmitting computer system to a receiving computer system to display the visual material information on a display screen of a display device of each of the transmitting and receiving computer systems. A computer-aided information transmitting method for transmitting voice knowledge information, which is described by voice while referring to visual material information displayed on a display screen, to a receiving computer system, wherein the sender and the receiving computer for the display screen in the transmitting computer system Computer-assisted information, wherein a line-of-sight direction of a receiver with respect to the display screen in the system is detected to obtain line-of-sight information of both, and a correlation state between the two line-of-sight information is displayed on a display screen of the transmitting computer system. Transmission method. 2. The computer-aided information transmission method according to claim 1, wherein an alarm is issued when the degree of coincidence of the binocular information becomes worse than a predetermined value. 3. The computer-assisted information transmission method according to claim 1, wherein the correlation state of the line-of-sight information is displayed in the order of goodness or badness. 4. A transmission-side computer system and a reception-side computer system are connected via a communication line, and visual data information is transmitted from the transmission-side computer system to the reception-side computer system so that the display devices of the computer systems on both sides of the transmission and reception are displayed. Computer-assisted display of visual material information on a screen, wherein a sender of a transmitting computer system transmits to a receiving computer system audio knowledge information described by voice while referring to the visual material information displayed on the display screen. In the information transmission device, the transmission-side computer system detects a line-of-sight direction of the sender with respect to the display screen to generate sender line-of-sight information, and transmission that transmits the sender line-of-sight information. User line-of-sight information transmitting means; A computer system comprising: sender line-of-sight information receiving means for receiving the sender line-of-sight information; and sender line-of-sight information display means for displaying the sender line-of-sight information on a display screen of a receiving computer system. Support information transmission device. 5. A receiver line-of-sight information generating means for detecting a line-of-sight direction of a receiver with respect to the display screen and generating receiver line-of-sight information according to claim 4, wherein the receiver line-of-sight information The transmission side computer system displays the recipient's line-of-sight information on the display screen of the transmission-side computer system. A computer-aided information transmission device, comprising: receiver line-of-sight information display means. 6. The computer-aided information transmission device according to claim 5, wherein said receiver line-of-sight information display means displays a correlation state between the sender line-of-sight information and the receiver line-of-sight information on a display screen. .