JP2006293038A - System, method, and computer program for supporting lecture - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a lecture support system wherein a teacher can effectively make learners take a review class or a remedial class of the lecture in the case the result of monitoring learner's state of receiving lecture is not satisfactory. <P>SOLUTION: The lecture support system comprises: an image storing means for making image data displayed in the lecture correlate with lapse time of the lecture and storing the image data; a means to set a flag on an arbitrary image data; a lecture receiving state storing means for making learner's state of receiving lecture correlate with lapse time of the lecture and storing the learner's state of receiving lecture; and a means to reproduce, when the learner's state of receiving lecture in time elapse of the lecture corresponding to the image data with the flag is at a prescribed level, the image data with the flag. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a lecture support system, a lecture support method, and a computer program for lecture support.

A system disclosed in Patent Document 1 is known as a system for supporting a lecture by grasping the attendance state of a student.
In this lecture support system, the attendance status of a plurality of students is determined based on the biometric information of the students. As a result of the judgment, for example, the ratio of the students who are concentrated in the lecture and the students who are not concentrated is output, and the lecturer is made aware of the situation of the student.
In addition, please refer to Patent Documents 2 to 4 that disclose techniques related to the present invention.
JP 2004-199506 A JP 2004-229948 A JP 2004-129703 A Japanese Patent Laid-Open No. 10-78743

According to the above lecture support system, the student's status is presented to the instructor, so the instructor can accent the lecture content and repeat or change the content of the lecture according to the student's situation. . As an accent, for example, when the lecture is unidirectional, it is possible to ask the student a question and call attention, or for a long lecture, take a short break.
However, even if it is understood that the attendance status of the student is favorable, the system cannot be involved at all in reviewing or supplementing.

  Therefore, the present invention has an object to provide a lecture support system that can effectively perform review and supplementary learning when the attendance state of the attendee is not favorable as a result of monitoring the attendance state of the attendee. To do.

The first aspect of the present invention has been made to achieve the above object. That is, image storage means for storing the image data displayed in the lecture in association with the elapsed time of the lecture,
Means for flagging any of the image data;
Attendance state storage means for storing the attendance state of the student in association with the elapsed time of the lecture,
Means for reproducing the image data with the flag when the attendance state of the student is at a predetermined level in the elapsed time of the lecture corresponding to the image data with the flag;
Lecture support system with

  According to the lecture support system configured as described above, for example, when the attendance state of the students is not preferable in the elapsed time zone of the lecture in which the image data with the flag indicating that the importance is high is displayed It is highly likely that the lecture content regarding the image data is not fully understood by the students. Therefore, the lecture content can be reviewed by reproducing the image data. According to this system, an important part of a lecture and a part where the student's concentration is insufficient are automatically extracted, and the image data of the part is reproduced, so that it can be efficiently performed in a short time. Review and supplementary lessons can be conducted.

The second aspect of the present invention is defined as follows. That is,
Voice storage means for storing lecture voice data in association with the elapsed time of the lecture;
Means for reproducing audio data corresponding to the elapsed time of a lecture corresponding to the image data from the audio storage unit when reproducing the flagged image data, and reproducing the data at high speed;
Lecture support system characterized by that.

  According to the lecture support system of the second aspect configured as described above, when the image data for review required extracted by the system specified in the first aspect is reproduced, the recorded data of the lecture related thereto is reproduced at high speed. Is done. Thereby, review or supplementary lessons can be performed in a shorter time.

The third aspect of the present invention is defined as follows. That is, auxiliary data storage means for storing auxiliary data for review corresponding to the image data is further provided,
When reproducing the image data, the auxiliary data is read out from the auxiliary data storage means and output.

  According to the lecture support system of the third aspect configured in this way, auxiliary data such as quizzes and reference materials is output when replaying image data and reviewing lectures, so that the learning effect is improved. It will be even higher.

The lecture support system according to the fourth aspect of the present invention is defined as follows. That is, pointer position storage means for storing the relationship between the position of the pointer displayed in the image data and the elapsed time;
Means for specifying the elapsed time in relation to the position of the pointer displayed in the reproduced image data in relation to the relationship stored in the pointer position storage means, and reproducing the audio data from the specified elapsed time; Are further provided.

  According to the lecture support system of the fourth aspect configured as described above, when a pointer is placed at a desired position in the reproduced image data, an elapsed time when the pointer is placed at the position in the lecture is specified, and the progress The audio data of the lecture is played from time. As a result, the review target can be specified in detail, and the review learning efficiency is improved by avoiding unnecessary review.

Next, each element of the present invention will be described in detail.
The image data is a so-called slide used for a lecture including characters, symbols, graphs, diagrams and the like. As this slide, it is preferable to use an electronic slide such as PowerPoint (trade name). Of course, hard-copy slides can also be used.
The elapsed time of the lecture can be added as electronic data to the digitized slide. The hard-copied slide is once photographed with an imaging device such as a CCD and digitized to form image data, and the elapsed time is added as electronic data.

The image data can be flagged. For example, a flag of importance (high) is attached to image data displaying important contents in a lecture. In this embodiment, two flags (high) and (low) are attached, but the type and number of flags can be arbitrarily set.
The timing used in the lecture room, that is, the timing when the image data is displayed in the lecture room is stored in association with the elapsed time of the lecture.

Furthermore, in another aspect of the present invention, when an arbitrary position is indicated with a pointer in the image data, the indicated position is stored in association with the elapsed time of the lecture.
Furthermore, as defined in other aspects, it is preferable to form review auxiliary data corresponding to the image data.
This auxiliary data includes quizzes, reference materials, and the like that are effective for reviewing or supplementing the contents of image data. It is not necessary to provide this auxiliary data corresponding to all the image data. For example, the auxiliary data can be prepared only for auxiliary data to which the flag of importance (high) is attached. Also, a plurality of auxiliary data can be prepared for the image data. In this case, auxiliary data can be selected according to the attendance level of the student when the image data is used. For example, when the attendance level of the student is relatively good (level I or level II), the auxiliary data is displayed in a small test format, and in other cases, reference data is displayed as auxiliary data.
The output format of the auxiliary data can be arbitrarily selected according to the content. For example, in the case of a small test, it can be printed out on a screen display or paper. In the reference material, a sub-screen can be formed on the image data display screen, and the reference material can be displayed on the sub-screen.

The attendance state of the student can be obtained by detecting the biometric information of the student by a well-known method. A sensor that contacts the student can detect an electroencephalogram, heartbeat, electrocardiogram, skin potential, body temperature, and the like. Also. The position of the head or back can also be detected by a position sensor. An acceleration sensor that differentiates the output from the position sensor can also be used. Furthermore, the biometric information of the student can be detected by imaging the student's posture and processing the image.
In order to more accurately identify the attendance state of the student, it is preferable to use the detection results of the two or more pieces of biological information.

The attendance status of the students is, for example, (1) a state where concentration on the lecture is substantial (attendance state level I), (2) a state where concentration on the lecture is insufficient (attendance state level II), (3) Other than the lecture, the state is focusing (learning state level III), (4) sleeping (learning state level IV), (5) abnormal state due to some other reason (learning level V) be able to.
The detection result of each said biometric information in said each attendance state level (I)-(V) is beforehand preserve | saved as standard biometric information about a standard student. Then, the student's attendance state level is classified based on the stored standard biometric information on the actual biometric information detection result obtained for the student when the system of the present invention is executed. At this time, it is preferable to process a plurality of types of detection results to identify the attendance state level.

  In this way, by classifying the attendance state of each student for each attendance state level, a list of attendance state levels of all the learners who have been detected at a specific time is formed. That is, the distribution of the attendance level of each student at a specific time is obtained. In other words, a group of attendance state levels of a plurality of students (groups) can be regarded as an information unit to be processed.

  The attendance level of each student and / or the attendance level of the group as a group is specified at a predetermined timing and stored in association with the elapsed time of the lecture.

Lecture audio data is also stored in association with the elapsed time of the lecture.
This audio data can be reproduced from any elapsed time.
For high-speed playback, see, for example, "Development of a high-quality real-time speech speed conversion system", IEICE Transactions, Vol. J84-D-II Num. 6 pp.918-926 (2001.06) It is preferable to use technology.

  A general-purpose memory (RAM) or hard disk can be used for the image storage means, attendance state storage means, audio storage means, and pointer position storage means. These storage devices may be provided in the lecture support system or may be used as external storage devices.

Examples of the present invention will be described below.
FIG. 1 is a block diagram showing the configuration of a lecture support system 1 according to an embodiment of the present invention.
As shown in FIG. 1, the lecture support system 1 according to the embodiment is roughly composed of a terminal device 2 on the lecturer side and a sensor device 20 on the student side.
The instructor-side terminal device 2 includes a computer device, and various input / output devices, data storage devices, and interface devices are connected to a bus line connected to the central processing unit 3.

The input device 4 captures data into a computer device, and includes a keyboard, a mouse, a scanner, a recording medium reader, and the like.
The image data is stored in the image data storage unit 111 of the image storage device 11 via the input device 4. In this embodiment, the importance of the image data is also input from the input device 4 and stored in the importance storage unit 112 of the image storage device 11 in association with the image data.
Auxiliary data composed of small tests and reference materials is also stored in the auxiliary data storage unit 114 of the image storage device 11 in association with the image data via the input device 4.
The lecturer can input various parameters necessary for operating the lecture support system 1 using the input device 4. For example, a parameter or threshold value related to biometric information for determining the attendance state level is determined, or a person who detects the attendance state is specified.

The image data output device 5 enables a student to visually recognize image data such as a slide used in a lecture, and a projector, a television, or the like can be used. The image data can include a moving image.
The pointer 6 designates a desired part in the image data. For example, when characters, figures, symbols, etc. are entered in image data with an electronic pen, the position of the electronic pen in the image data is specified.
The instructor monitor 7 is a display composed of liquid crystal or CRT including a small device such as a PDA, and displays the output result of the lecture support system. A printer or an audio output device can be used instead of or in combination with the instructor monitor 7.

The main storage device 8 stores a control program, and the central processing unit 3 reads the control program from the main storage device 8 and controls the entire system.
The clock 9 is a standard device provided in the computer device, and the elapsed time of the lecture is determined based on the time of the clock 9.

  The image data acquisition apparatus 10 specifies the time when image data such as a slide used for a lecture is displayed in light of the lecture elapsed time generated by the clock 9. The image data is stored in the image data storage unit 111 of the image storage unit 11, and the relationship between the image data and the lecture elapsed time is stored in the time storage unit 113.

  The audio data acquisition device 12 records the audio data of the lecture (the instructor's real voice, the used sound effect, etc.) in light of the lecture elapsed time generated by the clock 9. The audio data is stored in the audio data storage unit 131 of the audio storage unit 13, and the relationship between the audio data and the lecture elapsed time is stored in the time storage unit 132.

  The position data acquisition device 14 specifies the position of the pointer 6 in the image data in light of the lecture elapsed time generated by the clock 9. The specified position is stored in the position data storage unit 151 of the pointer position storage device 15, and the relationship between the pointer position and the lecture elapsed time is stored in the time storage unit 152.

The attendance state identification device 16 processes outputs from various sensors of the student side sensor device 20 and identifies the attendance state of the learner.
The student side sensor device 20 includes a smart chair 21. The smart chair 21 includes various sensors, a headset, and a network interface on the chair body. A thermometer 22, a CCD camera 23, and an electrode 24 are provided as sensors. The thermometer 22 and the electrode 24 are brought into contact with the student, and the body temperature and the amount of perspiration are detected. Since the resistance of the entire body changes when the amount of sweating changes, the amount of sweating can be specified by measuring the resistance of the entire body using the electrode 24. The state of blinking can be specified by imaging a student's face, particularly around the eyes, with a CCD camera provided in the headset, and performing image processing using a known method. The brain wave of the student is measured by the electroencephalograph 25, and the movement of the student on the chair is detected by the position sensor 26. The personal authentication device 27 authenticates who is sitting on the smart chair. As this personal authentication device, a headset CCD camera can be used to authenticate the student from the photographed face and retina data. The pulse rate sensor 28 measures the heart rate of the student, and the blood sensor 29 measures the blood saturated oxygen concentration (SpO ₂ ) to measure the degree of fatigue of the student.
In the lecture room, all seats where the students sit may be smart chairs, and these sensors 22 to 29 may be attached, or only specific seats may be smart chairs.

体温については検出された温度がＴ１（温度幅があるものとする、Ｔ２〜Ｔ４についても同じ）のとき受講状態レベルはレベル１にあるもと標準化されている。
他方、瞬きの回数（単位時間当たり）がＮ１（幅があるものとする、Ｎ２〜Ｎ４についても同じ）のとき受講状態レベルはレベル１にあるものと標準化されている。
発汗量は受講者の体全体の電気抵抗（インピーダンス）がＲ１（幅があるものとする、Ｒ２〜Ｒ４についても同じ）のとき受講状態レベルはレベル１にあるものとする。
なお、実際の測定にあたっては検出対象の優先順位を付すことが好ましい。例えば、体温がＴ１であるにもかかわらず、瞬きの回数がＮ２となることがあるからである。優先順位を付した場合には、優先順位の高い生体情報の検出結果に基づき受講状態レベルが特定されることとなる。
この実施例では１つの生体情報の検出値と受講状態レベルとを関連付けているが、複数の生体情報の検出値の組合わせに受講状態レベルを関連付けることもできる。 In this embodiment, as shown in Table 1 below, the body temperature, blinking, and sweating amount are specified as standard biometric information, and the student's attendance state is specified as a detection target.

Regarding the body temperature, when the detected temperature is T1 (assuming that there is a temperature range, the same applies to T2 to T4), the attendance state level is standardized as being at level 1.
On the other hand, when the number of blinks (per unit time) is N1 (assuming there is a width, the same applies to N2 to N4), the attendance state level is standardized as being at level 1.
As for the amount of sweating, the attendance state level is assumed to be level 1 when the electrical resistance (impedance) of the entire body of the student is R1 (assuming there is a width, the same applies to R2 to R4).
In the actual measurement, it is preferable to give priority to detection targets. For example, even if the body temperature is T1, the number of blinks may be N2. When priorities are assigned, the attendance state level is specified based on the detection result of the biological information having a high priority.
In this embodiment, the detection value of one biological information and the attendance state level are associated with each other, but the attendance state level may be associated with a combination of detection values of a plurality of biological information.

The relationship of Table 1 is stored in a predetermined area 173 of the attendance state storage device 17.
The attendance state identification device 16 detects the student's biometric information and identifies the attendance state level of the attendee in light of the information shown in Table 1. The attendance elapsed time at the time of detection is specified from the time generated by the clock 9. The attendance state level specified in this way is stored in the attendance state data storage unit 171, and the relationship between the attendance state level and the lecture elapsed time is stored in the time storage unit 172.

The monitor 31 displays image data (slides) to students, and also displays auxiliary data such as quizzes and reference materials.
The monitor 31 and the smart chair 21 are connected to the network interface 18 via a wired or wireless network.

FIG. 2 shows a history of reference lecture contents. The slides 41-45 displayed along the elapsed time of the lecture are each assigned importance, and those with higher importance are indicated with (H) in the figure, and those with lower importance are indicated with (L). The start time of each slide is t11, t21, t31, t41, t51, and the end time is represented by t12, t22, t32, t42, t52. The slide is stored in the image data storage unit 111 of the image storage device 11, the importance is similarly stored in the importance storage unit 112, and the start time and the end time are stored in the time storage unit 113.
Auxiliary data 51-53 is prepared for the slide of importance (H), and this auxiliary data 51-55 is stored in the auxiliary data storage unit 114 of the image storage device 11.

It is the position of the pointer that is displayed in coordinates during the slide. The elapsed time at each position is represented by t13, t23, t33, t43, and t53, respectively.
In the example of FIG. 2, only one position is displayed as coordinates in each slide, but a plurality of pointer positions can be stored in association with the elapsed time.

In the example of FIG. 2, the attendance level of one student is recorded along the elapsed time. In this example, the attendance state levels I and II and the attendance state levels III and IV are collectively set as determination targets, but the determination target of the attendance state level can be arbitrarily set. When the attendance state level is V, it is excluded from the processing target as an abnormal situation.
The state change at the attendance state level can be stored for each student.
It is also possible to estimate the attendance level of the entire student from the attendance state level obtained for each student or a specific attendee, and to support review lectures for the entire student.
Audio data is also stored along the elapsed time.

Next, the operation of the lecture support system according to the embodiment will be described.
First, as shown in FIG. 3, in step 1, the slide is stored as image data in the image data storage unit 111 of the image storage device 11 via the input device 4. Similarly, the importance of each slide is stored in the importance storage unit 112. In step 3, auxiliary data is stored in the auxiliary data storage unit 114 via the input device 4.

In step 5, the lecture history is recorded. Details are shown in FIG.
In the lecture, the lecturer operates the computer device to read the slide stored in the image data storage unit 111 and displays it on the screen in the lecture room using the image data output device 5 and also displays it on the monitor 31 on the student side.
For the displayed slide, the display start time and the display end time (tx1, tx2) are acquired by the image data acquisition device 10 and stored in the time storage unit 113 (step 51).
In this embodiment, the image data stored in the image data storage unit prepared in advance was used. However, during the lecture, the data was converted into image data using a photographing projector or the like. In this case, such image data is acquired by the image data acquisition device 10 and stored in the image data storage unit 111, and the display start time and display end time (tx1, tx2) are stored in the time storage unit 113. . The degree of importance to be attached to the image data can be designated by the instructor via the input device 4. Further, the importance level is automatically extracted by using the method described in “Extraction of Slide Importance in Lecture Reproduction System”, Journal of the Japanese Society for Artificial Intelligence, Vol. 17, No. 4, SP-E (2002). 112 may be stored.

  In step 53, the audio data storage device 12 acquires the audio data of the lecture and stores it in the audio data storage unit 131 of the audio storage device 13, and the relationship between the audio data and the lecture elapsed time is stored in the time storage unit 132. The

  In step 55, when the lecturer uses a pointer on the slide during the lecture, the position designated by the pointer is saved in the position data saving unit 151 of the pointer position saving device 15, and the position data and the lecture elapsed time are The relationship is stored in the time storage unit 152.

  In step 57, the attendance state identification device 16 is used to identify the attendance level of the attendee's biometric information acquired by the attendee-side sensor device 20 according to the relationship shown in Table 1, and the attendance of the identified attendee The state level is stored in the attendance state data storage unit 171, and the relationship between the attendance state level and the lecture elapsed time is stored in the time storage unit 172.

Returning to FIG. 3, in step 7, based on the lecture history recorded in step 5, image data (slide) to be reproduced for review is extracted according to a predetermined rule.
As a predetermined rule, for example, when the importance of image data is H and the attendance level is III or IV in the majority of the lecture elapsed time in which the image data is displayed, the image data is Extract. In the example of FIG. 2, the image data 43 and the image data 45 are extracted.

  Thereafter, the extracted image data is output (step 9). Accompanying the output of the image data, audio data corresponding to the image data is reproduced. In the example of FIG. 2, the audio data of the lecture elapsed time from t31 to t32 is reproduced with the output of the image data 43, and the audio data of the lecture elapsed time from t51 to t52 with the output of the image data 45 is reproduced. Is played.

FIG. 5 shows a menu that the student can select.
That is, the student can select arbitrary image data from the image data extracted in step 7. This selection and subsequent selections are also made by using an input device such as a touch panel attached to the monitor 31.
In reproducing the audio data of the selected image data, it is possible to select whether or not the reproduction is performed at high speed (for example, double speed) (step 13). High-speed playback of audio data uses the technology described in "Development of a high-quality real-time speech speed conversion system", IEICE Transactions, Vol. J84-D-II Num. 6 pp.918-926 (2001.06) be able to.

  In step 17, the pointer position reproduction mode can be selected. When the pointer position reproduction mode is selected (step 18), the audio data related to the coordinates is reproduced by specifying the desired coordinates of the image data. In the example of FIG. 2, when the coordinates (x31, y31) in the image data 43 are designated, the audio data from the elapsed time t33 is reproduced.

  In step 21, the auxiliary data assist mode can be selected. When the auxiliary data assist mode is selected (step 22), the auxiliary data corresponding to the image data is read from the auxiliary data storage unit 114 and displayed on the monitor 31 on the student side. Auxiliary data includes quizzes and reference materials to improve the learning effect on the contents of image data.

  The present invention is not limited to the description of the embodiments and examples of the invention described above. Various modifications are also included in the present invention as long as those skilled in the art can easily conceive without departing from the scope of the claims.

FIG. 1 is a block diagram showing the configuration of a lecture support system according to an embodiment of the present invention. FIG. 2 is a time chart showing the operation of the lecture support system of the embodiment. FIG. 3 is a flowchart showing the operation of the lecture support system according to the embodiment. FIG. 4 is a flowchart illustrating the operation of the lecture support system according to the embodiment. FIG. 5 is a flowchart showing the operation of the lecture support system according to the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lecture support system 2 Instructor side terminal device 3 Central processing unit 11 Image storage device 13 Voice storage device 15 Pointer position storage device 17 Attendance state storage device 20 Student side sensor device 21 Chair 22 Thermometer 23 CCD
24 electrodes

Claims (6)

Image storage means for storing the image data displayed in the lecture in association with the elapsed time of the lecture,
Means for flagging any of the image data;
Attendance state storage means for storing the attendance state of the student in association with the elapsed time of the lecture,
Means for reproducing the image data with the flag when the attendance state of the student is at a predetermined level in the elapsed time of the lecture corresponding to the image data with the flag;
Lecture support system with Voice storage means for storing lecture voice data in association with the elapsed time of the lecture;
Means for reading out audio data corresponding to the elapsed time of the lecture corresponding to the image data from the audio storage means when reproducing the flagged image data, and for high-speed reproduction;
Lecture support system characterized by that. Auxiliary data storage means for storing auxiliary data for review corresponding to the image data is further provided,
3. The lecture support system according to claim 1, wherein when the image data is reproduced, the auxiliary data is read from the auxiliary data storage unit and output. 4. Pointer position storage means for storing the relationship between the position of the pointer displayed in the image data and the elapsed time;
Means for specifying the elapsed time in relation to the position of the pointer displayed in the reproduced image data in relation to the relationship stored in the pointer position storage means, and reproducing the audio data from the specified elapsed time; ,
The lecture support system according to claim 1, further comprising: Image storage means for storing the image data displayed in the lecture in association with the elapsed time of the lecture,
Means for flagging any of the image data;
Voice storage means for storing lecture voice data in association with the elapsed time of the lecture;
Means for reading out audio data corresponding to the elapsed time of the lecture corresponding to the image data from the audio storage means when reproducing the flagged image data, and for high-speed reproduction;
Lecture support system characterized by that. Image storage means for storing the image data displayed in the lecture in association with the elapsed time of the lecture,
Pointer position storage means for storing the relationship between the position of the pointer displayed in the image data and the elapsed time;
Voice storage means for storing lecture voice data in association with the elapsed time of the lecture;
The elapsed time of the lecture is specified in light of the relationship stored in the pointer position storage means with the position of the pointer displayed in the reproduced image data, and the audio data is reproduced from the specified elapsed time. Means,
Lecture support system comprising

Images