JP6672592B2 - Presenter selection support program, presenter selection support method, and information processing device - Google Patents

Description

  The present invention relates to a presenter selection support program, a presenter selection support method, and an information processing device.

  Conventionally, an assignment is transmitted from a teacher terminal to a student terminal, the student inputs an opinion on the assignment at the student terminal, and replies to the teacher terminal. There is a class form that makes the students announce their opinions.

  As prior art, for example, the attending attitude of a student of an Internet lecture is measured, and based on the measured attending attitude and the result of an understanding test conducted at the end of the lecture, the characteristics of electronic characters used in a computer game are determined. There is a technique for updating a constituent database. Further, the state of the plurality of students was determined based on the face images of the plurality of students extracted from images obtained by photographing the scenes of the plurality of students, and the face images of the plurality of students were determined. There is a technique for generating a list image for displaying a list with an index indicating the state of a student. Also, a technology for recognizing a student's face from a student image which is a moving image of one or more students' faces during a lecture, performing analysis on the recognized face, and outputting information related to the analysis result. There is.

JP-A-2004-301952 JP 2006-330364 A JP 2013-61906 A

  However, according to the related art, it may be difficult to select a presenter such as an opinion on an issue. For example, a student who fails to make a presentation may lose confidence and reluctant to do so. For this reason, teachers try to hit students who are as confident as possible while observing the students' complexion, but it is difficult to check each student's complexion during class to determine whether they are confident. It is.

  In one aspect, an object of the present invention is to provide a presenter selection support program, a presenter selection support method, and an information processing apparatus that support selection of a presenter.

  According to one aspect of the present invention, a shift instruction to a mode for selecting a presenter is received, and based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the shift instruction. A presenter selection support program, a presenter selection support method, which calculates a value indicating a ratio of the target person having the predetermined characteristic during the mode, and outputs the calculated value in association with the target person. And an information processing apparatus are proposed.

  According to one embodiment of the present invention, there is an effect that selection of a presenter can be supported.

FIG. 1 is an explanatory diagram of an example of a presenter selection support method according to an embodiment. FIG. 2 is an explanatory diagram illustrating a system configuration example of the education support system 200. FIG. 3 is an explanatory diagram showing an example of seat arrangement. FIG. 4 is a block diagram illustrating a hardware configuration example of the teacher terminal 201 and the like. FIG. 5 is an explanatory diagram showing an example of the stored contents of the student DB 220. FIG. 6 is an explanatory diagram illustrating an example of the storage content of the assignment DB 230. FIG. 7 is an explanatory diagram showing an example of the storage contents of the answer DB 240. FIG. 8 is an explanatory diagram showing an example of the storage contents of the gaze / depression flag table 250. FIG. 9 is an explanatory diagram showing an example of the storage contents of the first gaze / depression time record DB 260. FIG. 10 is an explanatory diagram showing an example of the storage contents of the second gaze / depression time record DB 270. FIG. 11 is an explanatory diagram showing a screen example of the assignment selection screen 1100. FIG. 12 is an explanatory diagram showing a screen example of the answer screen 1200. FIG. 13 is an explanatory diagram (part 1) of a screen example of the teacher matrix screen 1300. FIG. 14 is an explanatory diagram showing a screen example of the student matrix screen 1400. FIG. 15 is an explanatory diagram showing a screen example of the presentation screen 1500. FIG. 16 is a block diagram illustrating a functional configuration example of the teacher terminal 201. FIG. 17 is an explanatory diagram illustrating a specific example of a captured image. FIG. 18 is an explanatory diagram (part 2) of a screen example of the teacher matrix screen 1300. FIG. 19 is an explanatory diagram (part 1) of a screen example of a presenter selection screen. FIG. 20 is an explanatory diagram (part 2) of a screen example of the presenter selection screen. FIG. 21 is a flowchart illustrating an example of the educational support processing procedure of the teacher terminal 201. FIG. 22 is a flowchart illustrating an example of a specific processing procedure of the opinion collection processing. FIG. 23 is a flowchart illustrating an example of a specific processing procedure of the in-lecture processing. FIG. 24 is a flowchart (part 1) illustrating an example of a specific processing procedure of the during-presentation processing. FIG. 25 is a flowchart (part 2) illustrating an example of a specific processing procedure of the during-presentation processing. FIG. 26 is a flowchart (part 3) illustrating an example of a specific processing procedure of the during-presentation processing. FIG. 27 is a flowchart (part 1) illustrating an example of a specific processing procedure of the flagging processing. FIG. 28 is a flowchart (part 2) illustrating an example of a specific processing procedure of the flagging processing. FIG. 29 is a flowchart illustrating an example of a specific processing procedure of the face recognition processing. FIG. 30 is a flowchart illustrating an example of an opinion input processing procedure of the student terminal Ti.

  Hereinafter, embodiments of a presenter selection support program, a presenter selection support method, and an information processing apparatus according to the present invention will be described in detail with reference to the drawings.

(One embodiment of support method for presenter selection)
FIG. 1 is an explanatory diagram of an example of a presenter selection support method according to an embodiment. In FIG. 1, an information processing apparatus 101 is a computer that supports selection of a presenter. For example, the information processing apparatus 101 may be a server, a PC (personal computer), a notebook PC, a tablet PC, or the like.

  Further, the information processing apparatus 101 is connected to a camera 110 for photographing a subject, and can acquire an image photographed by the camera 110. The target person is a candidate for a presenter, for example, a student at a school or a student who receives some kind of lecture or course. Here, it is assumed that a student at a school is assumed as a target person, a class is given by a teacher to a student, and a class (so-called collaborative learning) for presenting an opinion on the subject to the student is performed.

  From an educational point of view, it is desirable to have various students present their opinions and gain experience. On the other hand, students who fail to make presentations may lose confidence and not want to make presentations in the future. For this reason, teachers want to give students the opportunity to gain a successful experience by focusing on the most confident students.

  Sometimes it is also important to make mistakes, so teachers may want to guess students who are less confident. For this reason, the teacher performs an action of hitting a student who seems to be as confident as possible or less confident while watching the complexion of the student in class. However, it is difficult for a teacher to check each student's complexion one by one to determine whether he / she is confident.

  Therefore, in the present embodiment, based on the image of the student taken while the teacher is selecting the presenter, an index value for determining whether or not the student is confident is obtained and presented to the teacher. A method for supporting the selection of the presenter that supports the selection of the presenter will be described. Hereinafter, a processing example of the information processing apparatus 101 will be described.

  (1) The information processing apparatus 101 receives an instruction to shift to the presenter selection mode. Here, the presenter selection mode is a mode for selecting a presenter. Specifically, for example, the information processing apparatus 101 receives an instruction to shift to the presenter selection mode by an operation input of a user (teacher).

  In addition, the teacher verbally informs the student that the presenter will be hit in response to the input operation of the instruction to shift to the presenter selection mode, for example. This tells the student that the presenter will be assigned. In addition, the opinion of the student may be displayed collectively on an electronic blackboard or the like in the classroom so that it can be understood that the presenter will be assigned in the future.

  (2) The information processing apparatus 101 performs face recognition processing on the image of the target person captured by the camera 110 after receiving the instruction to shift to the presenter selection mode. Here, the target person is at least one student who is a candidate for the presenter. Specifically, for example, the information processing apparatus 101 uses the learning data for face recognition to recognize the face of the target person from the image captured by the camera 110.

  The learning data for face recognition is generated, for example, from a face image obtained by capturing the face of the target person. For example, when learning data for face recognition is generated from an image captured in a state where the target person is facing forward, the face when the target person is facing forward can be recognized. When the learning data for face recognition is generated from an image taken in a state where the target person faces sideways (left or right), the face when the target person turns sideways can be recognized.

  In the example of FIG. 1, the subject is “Student A”, and an image of Student A taken at a predetermined time interval (for example, 0.1 second interval) by the camera 110 after receiving an instruction to shift to the presenter selection mode. It is assumed that face recognition processing is performed on D1 to D5. In FIG. 1, the t axis is a time axis indicating the passage of time.

  (3) The information processing apparatus 101 calculates a value indicating a ratio of the target person having the predetermined characteristic during the presenter selection mode, based on the recognition result of the recognition of the predetermined characteristic from the face image of the target person. Here, the predetermined feature is a feature for detecting, for example, a state in which the target person is facing down, or a state in which the subject faces a specific direction.

  For example, students who are more likely to look down while the teacher is trying to select a presenter are likely to be less confident in their opinion. On the other hand, while the teacher is trying to select a presenter, students who are more likely to look forward to the teacher are more confident in their opinion than if they are more likely to look down Probability is high.

  That is, during the presenter selection mode, the rate at which the student looks down or the rate at which the student looks up and faces the teacher is an index for determining whether the student is confident. Here, it is assumed that the teacher is located in front of the subject and the predetermined feature is “frontward state”. When the teacher is located on the left or right side of the subject, the predetermined feature may be “a state facing left or right”.

  In the example of FIG. 1, it is assumed that a face when the student A faces forward is recognized in the images D1 and D3 to D5 among the images D1 to D5. In this case, the information processing apparatus 101 obtains, for example, the ratio of the images D1 and D3 to D5 in which the face when the student A turns to the front is recognized, among the images D1 to D5, and thereby, during the presenter selection mode. The value indicating the ratio of the student A facing forward can be calculated.

  (4) The information processing apparatus 101 outputs a value indicating the calculated ratio having the predetermined feature in association with the subject. In the example of FIG. 1, a value “80 [%]” indicating the ratio of the student A facing forward during the presenter selection mode is output in association with the student name “A” of the student A.

  As described above, according to the information processing apparatus 101 according to the embodiment, based on the recognition result obtained by recognizing the predetermined feature from the face image of the target person captured after receiving the instruction to shift to the presenter selection mode, Thus, it is possible to calculate a value indicating the ratio of the subject having the predetermined feature during the presenter selection mode. Then, according to the information processing apparatus 101, the value indicating the ratio of the target person having the predetermined feature during the calculated presenter selection mode can be output in association with the target person.

  Thus, for example, an index value for determining whether or not a student has confidence in an opinion given to a task given by the teacher can be presented to the teacher, and it is possible to assist the teacher in selecting a presenter. In the example of FIG. 1, the teacher has a high value indicating the ratio of the student A facing forward during the presenter selection mode, “80 [%]”. It is possible to judge that it is highly likely to have, and it becomes easy to judge whether or not to hit the student A as the presenter.

(Example of system configuration of education support system 200)
Next, a case where the information processing apparatus 101 shown in FIG. 1 is applied to the teacher terminal 201 of the education support system 200 will be described.

  FIG. 2 is an explanatory diagram illustrating a system configuration example of the education support system 200. 2, the education support system 200 has a configuration including a teacher terminal 201, an electronic blackboard 202, and student terminals T1 to Tn. In the education support system 200, the teacher terminal 201, the electronic blackboard 202, and the student terminals T1 to Tn are connected via a wired or wireless network 210. The network 210 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

  Here, the teacher terminal 201 is a computer used by the teacher, and includes a student DB (database) 220, an assignment DB 230, an answer DB 240, a gaze / depression flag table 250, a first gaze / depression time record DB 260, and a second gaze. -It has a depression time record DB 270. The teacher terminal 201 is, for example, a PC, a notebook PC, a tablet PC, or the like. The storage contents of the various DBs 220, 230, 240, 250, 260, and 270 will be described later with reference to FIGS.

  The electronic blackboard 202 has a display (for example, a display 406 shown in FIG. 4 described later), and various screens (for example, a student matrix screen 1400 shown in FIG. 14 described later and FIG. 15) according to display control of the teacher terminal 201. On the presentation screen 1500). The electronic blackboard 202 is installed, for example, at a position visible to each student in the classroom.

  The student terminals T1 to Tn are computers used by the students S1 to Sn, respectively. The student terminals T1 to Tn are, for example, a tablet PC, a notebook PC, or the like. In the following description, an arbitrary student terminal among the student terminals T1 to Tn may be described as “student terminal Ti” (i = 1, 2,..., N). Further, an arbitrary student among the students S1 to Sn may be described as “student Si”.

(Example of seat arrangement)
Next, an example of arrangement of seats in a classroom will be described.

  FIG. 3 is an explanatory diagram showing an example of seat arrangement. FIG. 3 shows an example of the seat arrangement of the students S1 to S8 sitting on the platform in the classroom. The camera 300 is a photographing device that photographs the students S1 to S8 sitting on the seats. The camera 300 outputs, for example, a captured image captured at a predetermined time interval (for example, 1 [frame / 0.1 second]) to the teacher terminal 201 via an image I / F (not shown). The camera 300 corresponds to, for example, the camera 110 shown in FIG.

  The camera 300 may be provided on the teacher terminal 201, for example, or may be provided separately from the teacher terminal 201. Specifically, for example, the camera 300 may be a fixed-point camera installed on a teaching platform or the like in a classroom, or may be a wearable camera mounted on a teacher's head or the like. Further, the camera 300 may be an infrared camera.

  Here, the positions of the seats of the students S1 to S8 and the positions of the cameras 300 are determined in advance, and it is assumed that the teacher conducts a class at a predetermined position (for example, substantially the same position as the camera 300). I do.

(Example of hardware configuration such as teacher terminal 201)
Next, an example of a hardware configuration of the teacher terminal 201, the electronic blackboard 202, and the student terminal Ti (hereinafter, simply referred to as “teacher terminal 201 or the like”) illustrated in FIG. 2 will be described.

  FIG. 4 is a block diagram illustrating a hardware configuration example of the teacher terminal 201 and the like. 4, a teacher terminal 201 and the like include a CPU (Central Processing Unit) 401, a memory 402, a disk drive 403, a disk 404, an I / F (Interface) 405, a display 406, an input device 407, Having. Each component is connected by a bus 400.

  Here, the CPU 401 controls the entire control of the teacher terminal 201 and the like. The memory 402 includes, for example, a ROM (Read-Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, a flash ROM or a ROM stores various programs, and a RAM is used as a work area of the CPU 401. The program stored in the memory 402 is loaded on the CPU 401 to cause the CPU 401 to execute a coded process.

  The disk drive 403 controls reading / writing of data from / to the disk 404 under the control of the CPU 401. The disk 404 stores data written under the control of the disk drive 403. Examples of the disk 404 include a magnetic disk and an optical disk.

  The I / F 405 is connected to the network 210 through a communication line, and is connected to an external device through the network 210. The I / F 405 manages an internal interface with the network 210 and controls data input / output from an external device. The network 210 is, for example, a LAN, a WAN, the Internet, or the like.

  The display 406 displays data such as documents, images, function information, etc., in addition to a cursor, icons, or tool boxes. As the display 406, for example, a CRT (Cathode Ray Tube), a liquid crystal display, an organic EL (Electroluminescence) display, or the like can be employed.

  The input device 407 has keys for inputting characters, numerals, various instructions, and the like, and inputs data. The input device 407 may be a keyboard, a mouse, or the like, or may be a touch panel-type input pad, a numeric keypad, or the like.

  Note that the teacher terminal 201 and the like may not include, for example, the disk drive 403 or the disk 404 among the above-described components. The teacher terminal 201 and the like may include, for example, a camera 300 (see FIG. 3), an SSD (Solid State Drive), a scanner, a printer, and the like, in addition to the above-described components.

(Storage contents of 220, 230, 240, 250, 260, 270, etc.)
Next, the storage contents of various DBs 220, 230, 240, 250, 260, and 270 will be described with reference to FIGS. The various DBs 220, 230, 240, 250, 260, and 270 are realized by a storage device such as the memory 402 and the disk 404 of the teacher terminal 201 illustrated in FIG.

  FIG. 5 is an explanatory diagram showing an example of the stored contents of the student DB 220. In FIG. 5, the student DB 220 has fields of a student ID, a student name, a terminal address, and an area coordinate, and stores information of each of the fields as a record by setting information in each field.

  Here, the student ID (identification) is an identifier for identifying the student Si. The student name is the name of the student Si. The terminal address is an identifier for identifying the student terminal Ti used by the student Si, and is, for example, an IP (Internet Protocol) address of the student terminal Ti.

  The area coordinates indicate the position of the face area of the student Si in the image captured by the camera 300. The face area is an area where the face of the student Si is located on the captured image. Here, the area coordinates indicate the coordinate positions of the upper left, upper right, lower left, and lower right vertices of the face area (rectangle) of the student Si in the image coordinate system including the X axis and the Y axis.

  For example, the student information 500-1 includes the student name “Tanaka” of the student S1, the terminal address “XX.YY.ZZ.A1”, and the area coordinates “(21,58) (30,58) (21,40) (30) , 40) ".

  FIG. 6 is an explanatory diagram illustrating an example of the storage content of the assignment DB 230. In FIG. 6, the assignment DB 230 includes fields of an assignment ID, an assignment content, an answer time limit (minutes), a subject, and a difficulty level. Store as a record.

  Here, the assignment ID is an identifier for identifying an assignment given to a student by a teacher. The task content is the content of the task. The answer time limit (minutes) is the time limit from giving an assignment to a student to receiving an answer. The subjects distinguish the areas that correspond to the tasks. The subjects include, for example, Japanese language, mathematics, society, and science.

  The difficulty level is a degree indicating the difficulty level of the task. Here, the number indicating the degree of difficulty is set to be larger for a more difficult task. For example, the assignment information 600-1 indicates the assignment content of the assignment K1, "What do you think about abortion of a minor?", The answer time limit "20 minutes", the subject "Society", and the difficulty level "4".

  FIG. 7 is an explanatory diagram showing an example of the storage contents of the answer DB 240. In FIG. 7, the answer DB 240 has fields of assignment ID, student ID, approval / disapproval, and opinion content, and sets information in each field to store answer information 700-1 to 700-8 as records.

  Here, the assignment ID is an identifier for identifying the assignment. The student ID is an identifier for identifying the student Si. Pros and cons indicate whether student Si agrees or disagrees with what was asked in the assignment. The opinion content is the content of the student's opinion on the assignment. For example, the answer information 700-1 indicates that the student S1 agrees or disagrees with the assignment K2 and agrees with the content of the opinion "If you are in trouble, you should help."

  FIG. 8 is an explanatory diagram showing an example of the storage contents of the gaze / depression flag table 250. In FIG. 8, the gaze / depression flag table 250 records a gaze flag, a depression flag, and a state flag of each of the students S1 to S8 for each recording date and time as a record.

  Here, the recording date and time is the date and time when each flag (fixation flag, depression flag, status flag) is recorded. The gaze flag is a flag indicating whether or not the student Si was looking at the teacher. The gaze flag is set to “1” when watching the teacher, and is set to “0” when not watching the teacher.

  The downward flag is a flag indicating whether or not the student Si is downward. The depression flag is set to “1” when the user is facing down, and is set to “0” when the user is not facing down. The status flag is a flag indicating the status of the lesson, and indicates, for example, any of a status during a lecture, a presenter being selected, and a presentation. In the status flag, “1” is set when a lecture is being given, “2” is set when a presenter is selected, and “3” is set when a presenter is giving a presentation.

  FIG. 9 is an explanatory diagram showing an example of the storage contents of the first gaze / depression time record DB 260. In FIG. 9, a first gaze / depression time record DB 260 stores various types of information on each of the students S1 to S8 in the presenter selection mode. The presenter selection mode is a mode in which the teacher can select a student to be a presenter during the presentation time, and corresponds to the state of the lesson when the state flag (see FIG. 8) is “2”.

  Specifically, in the past lecture accumulation column of the first gaze / depression time record DB 260, the cumulative presenter selection time (s) of each student S1 to S8 in the presenter selection mode in the past lecture, and the gaze time ( s), depression time (s), gaze rate (%) and depression rate (%) are recorded.

  Here, the cumulative presenter selection time (s) in the past lecture accumulation column indicates the cumulative time of the presenter selection mode in the past lecture. The gaze time (s) in the past lecture accumulation column indicates the accumulated time during which the student Si looked in the direction of the teacher during the presenter selection mode in the past lecture. The depression time (s) in the past lecture accumulation column indicates the accumulated time that the student Si has been depressed during the presenter selection mode in the past lecture.

  The gaze rate (%) in the past lecture accumulation column indicates a rate at which the student Si looked in the direction of the teacher during the presenter selection mode in the past lecture. The look-down ratio (%) in the past lecture accumulation column indicates the ratio of the student Si looking down during the presenter selection mode in the past lecture.

  Further, the current lecture column of the first gaze / depression time record DB 260 includes an update date and time, a selection time (s) of each of the students S1 to S8 in the presenter selection mode in the current lecture, a gaze time (s), The depression time (s), the gaze rate (%) and the depression rate (%) are recorded.

  Here, the update date and time is the date and time when the information in the current lecture section was updated. The selection time (s) in the current lecture column indicates the accumulated time of the presenter selection mode in the current lecture. The watching time (s) in the current lecture column indicates the accumulated time during which the student Si is looking in the direction of the teacher during the presenter selection mode in the current lecture.

  The look-down time (s) in the current lecture column indicates the accumulated time during which the student Si was looking down during the presenter selection mode in the current lecture. The gaze rate (%) in the current lecture column indicates the rate at which the student Si looked in the direction of the teacher during the presenter selection mode in the current lecture. The look-down rate (%) in the current lecture field indicates the rate at which the student Si is looking down during the presenter selection mode in the current lecture.

  FIG. 10 is an explanatory diagram showing an example of the storage contents of the second gaze / depression time record DB 270. In FIG. 10, a second gaze / depression time record DB 270 stores various types of information on the students S1 to S8 during the lecture. The lecture is in a state where the teacher is giving a lecture to the student, and corresponds to the state of the class when the state flag (see FIG. 8) is “1”.

  Specifically, the past lecture accumulation column of the second gaze / depression time record DB 270 includes the lecture time (m), the observation time (m), and the depression time of each of the students S1 to S8 in the past lecture. (M), gaze rate (%) and depression rate (%) are recorded.

  Here, the in-lecture time (m) in the past lecture accumulation column indicates the accumulated time during the lecture in the past lecture. The gaze time (m) in the past lecture accumulation column indicates the accumulated time during which the student Si looked in the direction of the teacher during the lecture in the past lecture. The depression time (m) in the past lecture accumulation column indicates the accumulated time that the student Si has been depressed during the lecture in the past lecture.

  The gaze rate (%) in the past lecture accumulation column indicates a rate at which the student Si looked in the direction of the teacher during the lecture in the past lecture. The look-down rate (%) in the past lecture accumulation column indicates the rate at which the student Si was looking down during the lecture in the past lecture.

  Further, the current lecture field of the second gaze / down time recording DB 270 includes the duration during lecture (m), the gaze time (m), the down time (m), the gaze rate (%), and the down rate (%). Be recorded.

  Here, the time during lecture (m) in the current lecture column indicates the accumulated time during the lecture in the current lecture. The gaze time (m) in the current lecture column indicates the accumulated time during which the student Si looked in the direction of the teacher during the lecture in the current lecture. The look-down time (m) in the current lecture column indicates the accumulated time during which the student Si was looking down during the lecture in the current lecture.

  The gaze rate (%) in the current lecture column indicates a rate at which the student Si looked in the direction of the teacher during the lecture in the current lecture. The look-down rate (%) in the current lecture column indicates the rate at which the student Si is looking down during the lecture in the current lecture.

(Screen example of the task selection screen 1100)
Next, a screen example of the assignment selection screen 1100 displayed on the display 406 of the teacher terminal 201 will be described. In the following description, a case where a click operation is performed as an operation for the user to select items and buttons displayed on various screens of the teacher terminal 201 will be described as an example.

  FIG. 11 is an explanatory diagram showing a screen example of the assignment selection screen 1100. In FIG. 11, an assignment selection screen 1100 is a screen for accepting selection of an assignment to be considered by a student.

  On the assignment selection screen 1100, when one of the selection buttons 1101 to 1103 is clicked by the user's operation input using the input device 407 (see FIG. 4) of the teacher terminal 201, the assignment to be considered by the student can be selected. it can. In the example of FIG. 11, the selection button 1102 is clicked, and the assignment K2 is selected.

  When the user clicks an assignment transmission button 1104 on the assignment selection screen 1100, information on the selected assignment (eg, assignment content of the assignment K2) can be transmitted from the teacher terminal 201 to the student terminals T1 to Tn. As a result, for example, an answer screen 1200 as shown in FIG. 12 described later is displayed on the display 406 of each of the student terminals T1 to Tn.

  When the user clicks a cancel button 1105 on the assignment selection screen 1100, the display of the assignment selection screen 1100 can be ended. In addition, the assignment selection screen 1100 may display the difficulty of each assignment together.

(Screen example of the answer screen 1200)
Next, a screen example of the answer screen 1200 displayed on the display 406 of the student terminal Ti will be described. In the following description, a case where a tap operation is performed as an operation for the user to select items and buttons displayed on various screens of the student terminal Ti will be described as an example.

  FIG. 12 is an explanatory diagram showing a screen example of the answer screen 1200. In FIG. 12, an answer screen 1200 is a screen for receiving an input of an opinion content on a task given by a teacher. In the example of FIG. 12, the answer screen 1200 displays the task content of the task K2, "What do you think about receiving rubble from the disaster area?"

  On the answer screen 1200, when one of the check boxes 1201 and 1202 is tapped by the user's operation input using the input device 407 (see FIG. 4), whether or not the user is in agreement or disagree with the question asked in the assignment Can be selected. In the example of FIG. 12, the check box 1201 is in a tapped state.

  Further, in the answer screen 1200, when the opinion column 1203 is tapped, it is possible to input the contents of the opinion on the assignment. In the example of FIG. 12, the content of the opinion on the assignment has not been entered in the opinion column 1203, but has not been entered. In addition, when the opinion clear button 1204 is tapped on the answer screen 1200, the content input in the opinion column 1203 can be cleared.

  Further, when the transmission button 1205 is tapped on the response screen 1200, the response result can be transmitted from the student terminal Ti to the teacher terminal 201. The answer result includes, for example, a selection result of approval or disapproval of the question asked in the assignment, and the opinion content input in the opinion column 1203.

  As a result, for example, a teacher matrix screen 1300 as shown in FIG. 13 described later is displayed on the display 406 of the teacher terminal 201. Also, when the response result is transmitted from the student terminal Ti to the teacher terminal 201, the display result 406 of the student terminal Ti is, for example, a selection result of the approval or disapproval of the question asked in the assignment, and is input to the opinion column 1203. The opinion content is displayed.

(Screen example of the matrix screen for teacher 1300)
Next, a screen example of the teacher matrix screen 1300 displayed on the display 406 of the teacher terminal 201 will be described.

  FIG. 13 is an explanatory diagram (part 1) of a screen example of the teacher matrix screen 1300. In FIG. 13, a teacher matrix screen 1300 is a screen that displays the content of the student's opinion on the assignment given by the teacher. In the example of FIG. 13, the opinions 1301 to 1308 of the student regarding the assignment K2 are displayed.

  For example, the opinion content 1301 indicates the student S1's approval / disapproval for the assignment K2, "Agree", the opinion content "Help me when I am in trouble", and the student name "Tanaka".

  When the presentation mode transition button 1310 is clicked on the teacher matrix screen 1300, the teacher terminal 201 transitions to the presenter selection mode. Further, by clicking the class end button 1320 on the teacher matrix screen 1300, the display of the teacher matrix screen 1300 can be ended.

  The display contents of the teacher matrix screen 1300 are updated each time the teacher terminal 201 receives an answer result from the student terminal Ti, for example. In the matrix screen for teacher 1300, the display position of the opinion content of each student is set in advance according to, for example, the seat position of each student.

(Student matrix screen 1400)
Next, a screen example of the student matrix screen 1400 displayed on the display 406 of the electronic blackboard 202 will be described.

  FIG. 14 is an explanatory diagram showing a screen example of the student matrix screen 1400. In FIG. 14, a matrix screen for students 1400 is a screen that displays a list of the opinions of the students regarding the assignments given by the teacher. In the example of FIG. 14, the opinion contents 1401 to 1408 of the student with respect to the assignment K2 are displayed.

  The opinion contents 1401 to 1408 are the same as the opinion contents 1301 to 1308 displayed on the teacher matrix screen 1300 shown in FIG. According to the student matrix screen 1400, each of the students S1 to S8 can confirm the content of the opinion of another student on the assignment K2. For example, when the student matrix screen 1400 is displayed on the electronic blackboard 202, the teacher starts a lecture on the assignment, and then selects a presenter of an opinion on the assignment.

(Screen at presentation 1500)
Next, a screen example of the presentation screen 1500 displayed on the display 406 of the electronic blackboard 202 will be described.

  FIG. 15 is an explanatory diagram showing a screen example of the presentation screen 1500. In FIG. 15, a presentation screen 1500 is a screen for enlarging and displaying the opinion of the student Si selected as the presenter. In the example of FIG. 15, the opinion content 1510 (pros and cons, opinion content) of the student S5 (Mr. Kuroda) selected as the presenter of the opinion on the assignment K2 is enlarged and displayed.

  According to the presentation screen 1500, each of the students S1 to S8 can confirm the content of the presenter's opinion on the assignment K2. In addition, when the presentation end button 1520 is clicked on the presentation screen 1500, the display of the presentation screen 1500 can be ended. The input operation on the presentation screen 1500 can be performed using, for example, the input device 407 of the teacher terminal 201.

(Functional configuration example of teacher terminal 201)
FIG. 16 is a block diagram illustrating a functional configuration example of the teacher terminal 201. 16, the teacher terminal 201 includes an acquisition unit 1601, a reception unit 1602, a display control unit 1603, a transmission unit 1604, a reception unit 1605, a recognition unit 1606, a detection unit 1607, a setting unit 1608, The configuration includes a calculation unit 1609 and an update unit 1610. The acquisition unit 1601 to the update unit 1610 are functions serving as a control unit. Specifically, for example, by causing the CPU 401 to execute a program stored in a storage device such as the memory 402 and the disk 404 illustrated in FIG. Implement that function. The processing result of each functional unit is stored in a storage device such as the memory 402 and the disk 404, for example.

  The acquisition unit 1601 acquires a captured image captured by the camera 300. Here, the photographed image is, for example, an image photographing the students S1 to S8 sitting on the seats. Specifically, for example, the obtaining unit 1601 obtains, from the camera 300, an image captured by the camera 300 at [1 frame / 0.1 second]. A specific example of the captured image will be described later with reference to FIG.

  The receiving unit 1602 receives a request to display the assignment selection screen 1100 (see FIG. 11). Specifically, for example, the receiving unit 1602 receives a display request for the assignment selection screen 1100 by a user's operation input using the input device 407 of the teacher terminal 201.

  The display control unit 1603 displays the assignment selection screen 1100 on the display 406 of the teacher terminal 201. Specifically, for example, in response to receiving a display request for the assignment selection screen 1100, the display control unit 1603 refers to the assignment DB 230 (see FIG. 6) to generate screen information of the assignment selection screen 1100. Thereby, the task selection screen 1100 is displayed on the display 406.

  As a result, an assignment selection screen 1100 that allows the student to select an assignment to be considered can be displayed on the display 406 of the teacher terminal 201.

  The receiving unit 1602 receives the selection of the assignment ID of the assignment that the student wants to consider. Specifically, for example, the accepting unit 1602 accepts selection of an assignment ID on the assignment selection screen 1100 by a user's operation input. In the following description, the assignment corresponding to the assignment ID for which the selection has been accepted may be referred to as “assignment K”.

  The display control unit 1603 controls the display of the answer screen 1200 (see FIG. 12) on the display 406 of the student terminals T1 to Tn. Specifically, for example, first, the display control unit 1603 extracts the task information of the task K from the task DB 230 in response to the click of the task transmission button 1104 on the task selection screen 1100. Next, the display control unit 1603 refers to the student DB 220 (see FIG. 5) and specifies the terminal addresses of the student terminals T1 to Tn. Then, the display control unit 1603 controls the transmission unit 1604 to transmit the extracted assignment information of the assignment K to the terminal addresses of the student terminals T1 to Tn.

  As a result, it is possible to display, on the display 406 of each of the student terminals T1 to Tn, an answer screen 1200 on which the content of an opinion on the assignment K given by the teacher can be input.

  The receiving unit 1605 receives the answer result for the assignment K from the student terminal Ti. The answer result includes, for example, an assignment ID, a student ID, a selection result of approval / disapproval for a question asked in the assignment K, an opinion content for the assignment K, and the like. The received answer result is stored, for example, in the answer DB 240 (see FIG. 7).

  The display control unit 1603 displays the teacher matrix screen 1300 (see FIG. 13) on the display 406 of the teacher terminal 201. Specifically, for example, the display control unit 1603 displays the teacher matrix screen 1300 on the display 406 by generating screen information of the teacher matrix screen 1300 with reference to the answer DB 240.

  Thereby, the teacher matrix screen 1300 on which the content of the opinion on the assignment K submitted from the student Si can be viewed on the display 406 of the teacher terminal 201 can be displayed.

  Receiving section 1602 receives an instruction to shift to the presenter selection mode. Specifically, for example, when the presentation mode transition button 1310 is clicked on the teacher matrix screen 1300 (see FIG. 13), the reception unit 1602 receives a transition instruction to the presenter selection mode.

  In addition, the reception unit 1602 performs voice recognition of a voice received by a microphone (not shown) of the teacher terminal 201, for example, a voice such as "I will designate a presenter." May be accepted. Further, the receiving unit 1602 may receive an instruction to shift to the presenter selection mode by detecting that a physical button (not shown) provided on the teacher terminal 201 has been pressed.

  The recognition unit 1606 performs face recognition processing on the image of the student Si. Specifically, for example, first, the recognizing unit 1606 extracts an image of the face region of the student Si from the image captured by the camera 300. Then, the recognition unit 1606 recognizes the student Si's face from the extracted image of the face area using the learning data for face recognition.

  The learning data for face recognition is generated, for example, from a face image of the face of the student Si. For example, learning data for face recognition can be generated by obtaining a HOG (Histograms of Oriented Gradients) feature amount based on the face image of the student Si and learning by AdaBoost or the like.

  The face recognition process performed by the recognition unit 1606 is performed on a captured image captured by the camera 300 at a predetermined time interval (for example, [1 frame / 0.1 second]).

  Here, specific processing contents of the recognition unit 1606 will be described with reference to FIG.

  FIG. 17 is an explanatory diagram illustrating a specific example of a captured image. 17, a captured image 1700 is an image of the students S1 to S8 captured by the camera 300. Dotted frames 1701 to 1708 in FIG. 17 indicate face areas of the students S1 to S8, respectively. The X axis and the Y axis in FIG. 17 are coordinate axes in the image coordinate system.

  Here, a case where the face of the student S1 is recognized will be described. In this case, the recognizing unit 1606 acquires the area coordinates of the student S1 from the student DB 220 (see FIG. 5). Next, the recognition unit 1606 extracts, from the captured image 1700, an image of the face area 1701 specified by the acquired area coordinates of the student S1. Then, the recognition unit 1606 recognizes the face of the student S1 from the extracted image of the face area 1701 using the learning data for face recognition of the student S1. In the example of FIG. 17, since the student S1 is facing down, the face of the student S1 is not recognized from the image of the face area 1701.

  Next, a case where the face of the student S3 is recognized will be described. In this case, the recognition unit 1606 acquires the area coordinates of the student S3 from the student DB 220 (see FIG. 5). Next, the recognizing unit 1606 extracts, from the captured image 1700, an image of the face area 1703 specified by the acquired area coordinates of the student S3. Then, the recognition unit 1606 recognizes the face of the student S3 from the extracted image of the face area 1703 using the learning data for face recognition of the student S3. In the example of FIG. 17, since the student S3 faces forward, the face of the student S3 is recognized from the image of the face area 1703.

  Returning to the description of FIG. 16, the detection unit 1607 detects the line of sight of the student Si based on the image of the eye of the student Si obtained by face recognition of the image of the student Si. Here, the eye of the student Si is the surface of the eyeball of the student Si, for example, the surface of the eyeball including the black eye and the white eye that is exposed in association with the movement of the eyelid.

  Specifically, for example, the detecting unit 1607 detects the line of sight of the student Si based on the position of the moving point with respect to the reference point specified from the image of the eye of the student Si. More specifically, for example, the detection unit 1607 may detect the line of sight (line-of-sight vector) of the student Si using the reference point as the “eye corner” and the moving point as the “iris”.

  Further, the detection unit 1607 may detect the line of sight (line-of-sight vector) of the student Si, for example, with the reference point being “corneal reflection” and the moving point being “pupil”. The corneal reflection indicates, for example, a position on the cornea of a reflected light generated by irradiating infrared rays to the student Si using an infrared LED (Light Emitting Diode) (not shown).

  In addition, about a gaze detection technique, for example, the following web page <URL: http: // jp. Fujitsu. com / group / labs / techinfo / techguide / list / eye-movements. html>.

  The setting unit 1608 sets a status flag. Here, the status flag is a flag indicating the status of the lesson, and indicates, for example, any one of a status during a lecture, a presenter being selected, and a presentation. During the lecture, the teacher is giving a lecture to the students S1 to Sn. During the selection of the presenter, it is in a state where the presenter of the opinion on the assignment K can be selected. During the presentation, the student Si selected as the presenter of the opinion on the assignment K is presenting. The status flag is stored in a storage device such as the memory 402 and the disk 404, for example.

  Specifically, for example, the setting unit 1608 sets the status flag to “1” in response to receiving the answer result for the assignment K from the student terminals T1 to Tn. The status flag “1” indicates that a lecture is being given. Further, the setting unit 1608 may set the status flag to “1”, for example, in response to the lapse of the answer limit time of the assignment K.

  The setting unit 1608 sets the status flag to “2” in response to, for example, receiving an instruction to shift to the presenter selection mode. The status flag “2” indicates that the presenter is being selected. In addition, the setting unit 1608 sets the status flag to “3” in response to selection of the presenter of the opinion on the assignment K. The status flag “3” indicates that the announcement is being made.

  Further, the setting unit 1608 sets the depression flag of the student Si based on the recognition result recognized by the recognition unit 1606. Here, when the face of the student Si is not recognized, it can be determined that the student Si is facing down. Therefore, when the face of the student Si is not recognized, the setting unit 1608 sets the depression flag “1” of the student Si.

  More specifically, for example, when the face of the student Si is not recognized, the setting unit 1608 associates the gaze flag “0” and the depression flag “1” of the student Si in association with the recording date and time and the status flag. Is recorded in the gaze / depression flag table 250 (see FIG. 8).

  As a result, the downward flag “1” indicating that the student Si is downward can be recorded in association with the recording date and time and the status flag. Note that the recording date and time may be the date and time when the gaze flag and the depression flag are recorded, or may be the date and time when the captured image to be subjected to the face recognition processing is captured.

  Further, the setting unit 1608 sets the gaze flag of the student Si based on the detection result detected by the detection unit 1607. Here, if the line of sight of the student Si is directed in a specific direction where the teacher is located, it can be determined that the student Si is looking at the teacher.

  For this reason, the setting unit 1608 determines, for example, whether or not the detected line of sight of the student Si is directed to a specific direction in which a preset teacher is located. Then, the setting unit 1608 sets the gaze flag “1” of the student Si when the line of sight of the student Si is in a specific direction.

  More specifically, for example, when the gaze of the student Si is directed to a specific direction, the setting unit 1608 associates the gaze flag “1” of the student Si with the look-down flag “ "0" is recorded in the gaze / depression flag table 250.

  Thereby, the gaze flag “1” indicating that the student Si was looking at the teacher can be recorded in association with the recording date and time and the status flag.

  The calculation unit 1609 calculates the depression rate of the student Si in the presenter's selection mode in the present lecture based on the recognition result recognized by the recognition unit 1606. Specifically, for example, the calculation unit 1609 refers to the gaze / depression flag table 250 and counts the number R1 of records of the status flag “2”. Next, the calculating unit 1609 refers to the gaze / depression flag table 250 and counts the depression flag number F1 of the depression flag “1” of the student Si of the status flag “2”. Then, the calculating unit 1609 calculates the downward ratio of the student Si in the presenter selection mode in the present lecture by dividing the counted downward flag number F1 by the number of records R1.

  As a result, it is possible to calculate the downturn rate indicating the rate at which the student Si is down during the presenter selection mode in the current lecture.

  The calculated depression rate of the student Si in the presenter selection mode in the current lecture is set, for example, in the current lecture column in the first gaze / depression time record DB 260 (see FIG. 9). Further, the counted number of records R1 is converted into the selected time and set in the current lecture field in the first gaze / down time recording DB 260. The counted number of depression flags F1 is converted into the depression time and set in the current lecture field in the first gaze / depression time record DB 260.

  Note that the number of records R1 can be converted into a selection time by, for example, multiplying the shooting interval of the camera 300 (for example, 0.1 [s]). In addition, the flag number F1 can be converted into a downward looking time by, for example, multiplying the shooting interval of the camera 300 (for example, 0.1 [s]).

  Further, the calculation unit 1609 calculates the gaze rate of the student Si in the presenter selection mode in the present lecture based on the detection result detected by the detection unit 1607. Specifically, for example, the calculation unit 1609 refers to the gaze / depression flag table 250 and counts the number R1 of records of the status flag “2”. Next, the calculation unit 1609 refers to the gaze / depression flag table 250 and counts the number F2 of gaze flags of the gaze flag “1” of the student Si of the state flag “2”. Then, the calculating unit 1609 calculates the gaze rate of the student Si in the presenter selection mode in the current lecture by dividing the counted number of gaze flags F2 by the number of records R1.

  Thereby, it is possible to calculate the gaze rate indicating the rate at which the student Si was looking at the teacher during the presenter selection mode in the current lecture.

  The calculated gaze rate of the student Si during the presenter's selection mode in the current lecture is set, for example, in the current lecture column in the first gaze / depression time record DB 260. Further, the counted number of gaze flags F2 is converted into a gaze time and set in the current lecture column in the first gaze / depression time record DB 260. The gaze flag number F2 can be converted into a gaze time by, for example, multiplying the shooting interval (for example, 0.1 [s]) of the camera 300.

  Further, the calculating unit 1609 may calculate the downward gaze rate of the student Si in the current lecture based on the recognition result recognized by the recognizing unit 1606. Specifically, for example, the calculation unit 1609 refers to the gaze / depression flag table 250 and counts the number R2 of records of the status flag “1”. Next, the calculating unit 1609 refers to the gaze / depression flag table 250 and counts the depression flag number F3 of the depression flag “1” of the student Si of the status flag “1”. Then, the calculating unit 1609 divides the counted number of depression flags F3 by the number of records R2 to calculate the depression ratio of the student Si who is currently lectured.

  As a result, it is possible to calculate the downward gaze rate indicating the rate at which the student Si gazes down during the current lecture.

  The calculated downward rate of the student Si during the current lecture is set, for example, in the current lecture field in the second gaze / downtime record DB 270 (see FIG. 10). Further, the counted number of records R2 is converted into a lecture time, and is set in the current lecture column in the second gaze / depression time record DB 270. The counted number of depression flags F3 is converted into the depression time and is set in the current lecture field in the second gaze / depression time record DB 270.

  Note that the number of records R2 can be converted into a lecture time by, for example, multiplying the shooting interval of the camera 300 (for example, 0.1 [s]). In addition, the flag number F3 can be converted into a prolonged time by multiplying the shooting interval of the camera 300 (for example, 0.1 [s]), for example.

  Further, the calculating unit 1609 calculates the gaze rate of the student Si who is currently giving a lecture, based on the detection result detected by the detecting unit 1607. Specifically, for example, the calculation unit 1609 refers to the gaze / depression flag table 250 and counts the number R2 of records of the status flag “1”. Next, the calculating unit 1609 refers to the gaze / depression flag table 250 and counts the number of gaze flags F4 of the gaze flags “1” of the student Si of the state flag “1”. Then, the calculation unit 1609 calculates the gaze rate of the student Si who is currently lectured by dividing the counted number of gaze flags F4 by the number of records R2.

  Thereby, it is possible to calculate the gaze rate indicating the rate at which the student Si was looking at the teacher during the current lecture.

  The calculated gaze rate of the student Si during the current lecture is set, for example, in the current lecture column in the second gaze / depression time record DB 270. In addition, the counted number of gaze flags F4 is converted into a gaze time and set in the current lecture column in the second gaze / depression time record DB 270. Note that the gaze flag number F4 can be converted into a gaze time by, for example, multiplying the shooting interval of the camera 300 (for example, 0.1 [s]).

  The display control unit 1603 outputs the calculated depression rate of the student Si in the presenter selection mode in the current lecture in association with the student Si. Specifically, for example, the display control unit 1603 displays on the teacher matrix screen 1300 the depression rates of the students S1 to S8 in the presenter selection mode in the current lecture together with the opinion contents 1301 to 1308. Is also good.

  In addition, for example, the display control unit 1603 sorts the student names of the students S1 to S8 in the descending rate in the presenter selection mode in the present lecture in the descending order (or in the descending order) and displays the sorted names on the display 406. It may be. At this time, the display control unit 1603 displays on the display 406 the names of the top N students (for example, N = 3) with a high (or low) look-down ratio in the presenter selection mode in the current lecture. Is also good.

  In addition, the display control unit 1603 outputs the calculated gaze rate of the student Si in the presenter selection mode in the current lecture in association with the student Si. Specifically, for example, the display control unit 1603 displays, on the teacher matrix screen 1300, the gaze rates of the students S1 to S8 in the presenter selection mode in the current lecture together with the opinion contents 1301 to 1308. Is also good.

  Also, for example, the display control unit 1603 sorts the student names of the students S1 to S8 in the order of the gaze rate in the presenter selection mode in the present lecture (in the order of high (or low)) and displays them on the display 406. It may be. At this time, the display control unit 1603 displays the student names of the top N students (for example, N = 3) whose gaze rate is high (or low) in the presenter selection mode in the current lecture on the display 406. It may be.

  Further, the display control unit 1603 may compare the look-down rate of the student Si in the presenter's selection mode in the present lecture with the calculated look-down rate in the past lecture. The downward gaze rate in the presenter selection mode in the past lecture corresponds to, for example, the downward gaze rate in the past lecture accumulation column of the first gaze / down time record DB 260.

  Then, the display control unit 1603 may output the identification information of the student Si in which the difference in the depression rate is equal to or more than the threshold α. The threshold value α is set in advance and stored in a storage device such as the memory 402 and the disk 404. The threshold value α is set, for example, to a value of about 30 to 70 [%].

  Note that the difference in the depression rate is, for example, a value obtained by subtracting the depression rate in the presenter's selection mode in the past lecture from the depression rate of the student Si in the presenter selection mode in the current lecture. In this case, the student Si whose depression rate has increased by the threshold α or more can be specified during the presenter selection mode in the current lecture, rather than during the presenter selection mode in the past lecture.

  In addition, the difference in the depression rate may be, for example, a value obtained by subtracting the depression rate of the student Si in the presenter selection mode in the present lecture from the depression rate in the presenter selection mode in the past lecture. In this case, the student Si whose depression rate has decreased by the threshold α or more can be specified during the presenter selection mode in the current lecture, rather than during the presenter selection mode in the past lecture.

  Specifically, for example, the display control unit 1603 may display, on the teacher matrix screen 1300, an image of the student Si whose difference in the depression rate is equal to or larger than the threshold α, together with the opinion content. At this time, the display control unit 1603 may display an image when the student Si is looking down together with the opinion content.

  It should be noted that an example of a screen in the case where the image of the student Si whose difference in the depression rate is equal to or larger than the threshold α is displayed together with the opinion content on the teacher matrix screen 1300 will be described later with reference to FIG.

  In addition, the display control unit 1603 may display the student Si having the largest difference in the downward ratio as the presenter candidate. Specifically, for example, the display control unit 1603 may display a presenter selection screen 1900 as shown in FIG.

  Further, the display control unit 1603 may compare the gaze rate of the student Si in the presenter lecture in the presenter lecture mode in the present lecture with the calculated gaze rate in the presenter lecture in the past lecture. The gaze rate in the presenter selection mode in the past lecture corresponds to, for example, the gaze rate in the past lecture accumulation column of the first gaze / depression time record DB 260. Then, the display control unit 1603 may output the identification information of the student Si whose difference in the gaze rate is equal to or more than the threshold α.

  Note that the difference in the gaze rate is, for example, a value obtained by subtracting the gaze rate in the presenter selection mode in the past lecture from the gaze rate of the student Si in the presenter selection mode in the current lecture. In this case, the student Si whose gaze rate has increased by the threshold α or more can be specified during the presenter selection mode in the current lecture, rather than during the presenter selection mode in the past lecture.

  Further, the difference in gaze rate may be, for example, a value obtained by subtracting the gaze rate of the student Si in the presenter selection mode in the present lecture from the gaze rate in the presenter selection mode in the past lecture. In this case, the student Si whose gaze rate has decreased by the threshold α or more can be specified during the presenter selection mode in the current lecture, rather than during the presenter selection mode in the past lecture.

  Specifically, for example, the display control unit 1603 may display, on the teacher matrix screen 1300, an image of the student Si whose difference in gaze rate is equal to or larger than the threshold α, together with the opinion content. At this time, the display control unit 1603 may display an image when the student Si is looking at the teacher together with the opinion content. Furthermore, the display control unit 1603 may display the images of the top N students whose difference in gaze rate is equal to or larger than the threshold α, together with the content of the opinion.

  It should be noted that an example of a screen in the case where the image of the student Si whose difference in gaze rate is equal to or more than the threshold α is displayed together with the opinion content on the teacher matrix screen 1300 will be described later using FIG.

  The display control unit 1603 may display the student Si having the largest difference in the gaze rate as a presenter candidate. Specifically, for example, the display control unit 1603 may display a presenter selection screen 1900 as shown in FIG.

  In addition, the display control unit 1603 may compare the depression rate of the student Si in the current lecture with the calculated depression rate of the student Si in the presenter selection mode in the current lecture. The look-down rate during the current lecture corresponds to, for example, the look-down rate in the current lecture section of the second gaze / gazing time record DB 270. Then, the display control unit 1603 may output the identification information of the student Si in which the difference in the depression rate is equal to or more than the threshold α.

  The difference in the depression rate is, for example, a value obtained by subtracting the depression rate in the current lecture from the depression rate of the student Si in the presenter selection mode in the current lecture. In this case, the student Si whose depression rate has increased by the threshold α or more can be specified in the presenter selection mode in the current lecture rather than in the current lecture.

  In addition, the difference in the depression rate may be, for example, a value obtained by subtracting the depression rate of the student Si in the presenter selection mode in the present lecture from the depression rate in the current lecture. In this case, the student Si whose depression rate has decreased by the threshold α or more can be specified in the presenter selection mode in the current lecture rather than in the current lecture.

  Specifically, for example, the display control unit 1603 may display, on the teacher matrix screen 1300, a face image of the student Si whose difference in the depression rate is equal to or larger than the threshold α, together with the opinion content. At this time, the display control unit 1603 may display an image when the student Si is looking down together with the opinion content. Further, the display control unit 1603 may display the images of the top N students whose difference in the downward ratio is equal to or more than the threshold α together with the content of the opinion.

  The display control unit 1603 may compare the gaze rate of the student Si in the current lecture with the calculated gaze rate of the student Si in the presenter selection mode in the current lecture. The gaze rate during the current lecture corresponds to, for example, the gaze rate in the current lecture column of the first gaze / depression time record DB 260. Then, the display control unit 1603 may output the identification information of the student Si whose difference in the gaze rate is equal to or more than the threshold α.

  The difference in the gaze rate is, for example, a value obtained by subtracting the gaze rate in the current lecture from the gaze rate of the student Si in the presenter selection mode in the current lecture. In this case, the student Si whose gaze rate has increased by the threshold α or more can be specified in the presenter selection mode in the current lecture rather than in the current lecture.

  Further, the difference in the gaze rate may be, for example, a value obtained by subtracting the gaze rate of the student Si in the presenter selection mode in the current lecture from the gaze rate in the current lecture. In this case, the student Si whose gaze rate has decreased by the threshold α or more can be specified in the presenter selection mode in the current lecture rather than in the current lecture.

  Specifically, for example, the display control unit 1603 may display, on the teacher matrix screen 1300, an image of the student Si whose difference in gaze rate is equal to or larger than the threshold α, together with the opinion content. At this time, the display control unit 1603 may display an image when the student Si is looking at the teacher together with the opinion content. Furthermore, the display control unit 1603 may display the images of the top N students whose difference in gaze rate is equal to or larger than the threshold α, together with the content of the opinion.

  In addition, the display control unit 1603 may compare the look-down rate in the past lecture with the calculated look-down rate of the student Si in the presenter selection mode in the current lecture. The downward gaze rate during the past lecture corresponds to, for example, the downward gaze rate in the past lecture accumulation column of the second gaze / down time recording DB 270. Then, the display control unit 1603 may output the identification information of the student Si in which the difference in the depression rate is equal to or more than the threshold α.

  Further, the display control unit 1603 may compare the gaze rate of the past lecture with the calculated gaze rate of the student Si in the presenter selection mode in the present lecture. The gaze rate during the past lecture corresponds to, for example, the gaze rate in the past lecture accumulation column of the first gaze / depression time record DB 260. Then, the display control unit 1603 may output the identification information of the student Si whose difference in the gaze rate is equal to or more than the threshold α.

  Further, the display control unit 1603 displays the student names of the top N students whose gaze rates in the presenter selection mode in the current lecture are equal to or more than the threshold β according to the degree of difficulty of the assignment K. Is also good. At this time, the display control unit 1603 may display an image of the student Si looking at the teacher together with the student name.

  Specifically, for example, when the difficulty of the task K is low (for example, difficulty = 1 or 2), the display control unit 1603 displays a presenter selection screen 2000 as shown in FIG. It may be. The threshold value β is set in advance and stored in a storage device such as the memory 402 and the disk 404. The threshold value β is set to, for example, a value of about 80 [%]. Further, the difficulty level of the assignment K can be specified, for example, from the assignment DB 230.

  In addition, the receiving unit 1602 receives a selection of a presenter of an opinion on the assignment K. Specifically, for example, the receiving unit 1602 receives a selection of a presenter of an opinion on the assignment K on a teacher matrix screen 1300 illustrated in FIG. 18 described below.

  The display control unit 1603 performs control to display the presentation screen 1500 (see FIG. 15) on the electronic blackboard 202 in response to receiving the selection of the presenter of the opinion on the assignment K. Specifically, for example, the display control unit 1603 refers to the answer DB 240 and generates screen information of the presentation screen 1500 that enlarges and displays the opinion content of the student Si selected as the presenter. Then, the display control unit 1603 transmits the generated screen information of the presentation screen 1500 to the electronic blackboard 202.

  As a result, the content of the opinion of the student Si selected as the presenter of the opinion on the assignment K can be enlarged and displayed on the electronic blackboard 202.

  The receiving unit 1602 receives a class end request. Here, the lesson end request is for notifying the end of the lesson. Specifically, for example, the receiving unit 1602 receives a lesson end request in response to a click on the lesson end button 1320 on the teacher matrix screen 1300.

  The update unit 1610 updates the storage contents of the first gaze / depression time record DB 260 in response to receiving the lesson end request. More specifically, for example, the updating unit 1610 determines the selection time, the gaze time, and the depression time of each student Si in the current lecture column of the first gaze / depression time record DB 260, and stores the selected time of each student Si in the past lecture accumulation column. It is added to the cumulative presenter selection time, watching time, and depression time.

  Then, the update unit 1610 updates the gaze rate of each student Si in the past lecture accumulation column by dividing the gaze time of each student Si in the past lecture accumulation column by the cumulative presenter selection time. The update unit 1610 updates the look-down rate of each student Si in the past lecture accumulation column by dividing the look-down time of each student Si in the past lecture accumulation column by the cumulative presenter selection time.

  Thus, the gaze rate and the look-down rate of each student Si in the presenter selection mode in the past lecture can be updated according to the end of the current class (lecture).

  Further, the update unit 1610 updates the storage contents of the second gaze / depression time record DB 270 in response to receiving the lesson end request. Specifically, for example, the update unit 1610 calculates the in-lecture time, the gaze time, and the depression time of each student Si in the current lecture section of the second gaze / depression time record DB 270, and the student Si in the past lecture accumulation section. Is added to the in-class time, gaze time, and staring time.

  Then, the update unit 1610 updates the gaze rate of each student Si in the past lecture accumulation column by dividing the gaze time of each student Si in the past lecture accumulation column by the lecture time. Further, the update unit 1610 updates the look-down rate of each student Si in the past lecture accumulation column by dividing the depression time of each student Si in the past lecture accumulation column by the lecture time.

  Thereby, the gaze rate and the downward gaze rate of each student Si in the past lecture can be updated in accordance with the end of the current class (lecture).

  In the above description, the teacher terminal 201 implements the functions of the functional units 1601 to 1610, but is not limited thereto. Specifically, for example, a server that can communicate with the teacher terminal 201 and the student terminals T1 to Tn may realize the functions of the functional units 1601 to 1610.

(Screen example of the matrix screen for teacher 1300)
Next, a description will be given of an example of a screen in the case where the image of the student Si whose difference in the look-down ratio or the gaze ratio is equal to or larger than the threshold α is displayed together with the opinion content on the teacher matrix screen 1300.

  FIG. 18 is an explanatory diagram (part 2) of a screen example of the teacher matrix screen 1300. In FIG. 18, the teacher matrix screen 1300 is an example of a screen displayed after the presentation mode transition button 1310 is clicked on the teacher matrix screen 1300 shown in FIG.

  On the teacher matrix screen 1300, the depression images 1801 and 1802 of the students S4 and S6 are superimposed on the opinion contents 1304 and 1306 of the students S4 (Yamamoto) and S6 (Honda). Here, the students S4 and S6 are the top two students whose depression rates have increased by the threshold α or more in the presenter lecture mode in the current lecture than in the presenter selection mode in the past lecture.

  The downward images 1801 and 1802 are images when the students S4 and S6 are downward. The downward images 1801 and 1802 are images (downward images) of the face regions of the students S4 and S6 used for face recognition. However, an image stored in advance in a storage device such as the memory 402 or the disk 404 may be used as the downward image. The numbers displayed on the left side of the depression images 1801 and 1802 are the ranks (first and second) indicating the height of the increase in the depression rate.

  In the teacher matrix screen 1300, the gaze images 1803 and 1804 of the students S5 and S8 are superimposed on the opinions 1305 and 1308 of the students S5 (Kuroda) and S8 (Asai). Here, the students S5 and S8 are the top two students whose gaze rates have increased by the threshold α or more in the presenter selection mode in the current lecture than in the presenter selection mode in the past lecture.

  The gaze images 1803 and 1804 are images when the students S5 and S8 are looking at the teacher. The gaze images 1803 and 1804 are images (gaze images) of the face regions of the students S5 and S8 used for face recognition. However, an image stored in advance in a storage device such as the memory 402 and the disk 404 may be used as the gaze image. The numbers displayed on the left side of the gaze images 1803 and 1804 are the ranks indicating the height of the increase in the gaze rate.

  According to the teacher matrix screen 1300, the teacher can determine, from the depression images 1801 and 1802, the students S4 and S6 who have a higher percentage of depression in the presenter selection mode than in the past lecture. . Therefore, the teacher can determine that the students S4 and S6 are less confident than usual, and can easily determine whether or not to apply the students S4 and S6 as the presenter.

  In addition, according to the teacher matrix screen 1300, the teachers S5, S8 from the gaze images 1803, 1804 are more likely to face themselves during the presenter selection mode than in the past lecture. Can be determined. Therefore, the teacher can determine that the students S5 and S8 are more confident than usual, and can easily determine whether or not to apply the students S5 and S8 as the presenter. At this time, the teacher can judge a more confident student from the expressions of the students S5 and S8 by comparing the gaze images 1803 and 1804, for example.

  Further, when any one of the opinion contents 1301 to 1308 is clicked on the teacher matrix screen 1300, a presenter of the opinion on the assignment K2 can be selected. For example, when the opinion content 1305 is clicked, the student S5 is selected as a presenter, and the presentation screen 1500 for enlarging and displaying the opinion of the student S5 is displayed on the electronic blackboard 202.

(Screen example of presenter selection screen)
Next, a screen example of the presenter selection screen pop-up displayed on the display 406 will be described with reference to FIGS. 19 and 20.

  FIG. 19 is an explanatory diagram (part 1) of a screen example of a presenter selection screen. In FIG. 19, a presenter selection screen 1900 is a screen from which a presenter of an opinion on an assignment K can be selected. On the presenter selection screen 1900, a downward image 1901 and a gaze image 1902 are displayed.

  The look-down image 1901 is an image of the student S4 (Yamamoto) having a higher look-down rate (the difference in look-down rate is maximum) in the presenter selection mode in the current lecture than in the presenter selection mode in the past lecture. The gaze image 1902 is an image of the student S5 (Kuroda) having a higher gaze rate (the difference in gaze rate is maximum) in the presenter selection mode in the current lecture than in the presenter selection mode in the past lecture. is there.

  Further, on the presenter selection screen 1900, by clicking any of the depression image 1901 and the gaze image 1902, the presenter of the opinion on the assignment K2 can be selected.

  According to the presenter selection screen 1900, the teacher can determine, from the downward image 1901, the student S4 who has the highest rate of depression during the presenter selection mode as compared to the past lecture. For this reason, the teacher can determine that the student S4 is considerably less confident than usual, and that he is a student who does not want it to be applied only this time. It becomes easy to determine whether or not.

  Further, according to the presenter selection screen 1900, the teacher determines, from the gaze image 1902, the student S5 who has the highest rate of facing himself / herself during the presenter selection mode compared to the past lecture. be able to. Therefore, the teacher can determine that the student S5 is much more confident than usual, and can determine that the student S5 wants to apply only this time. It becomes easy to judge.

  FIG. 20 is an explanatory diagram (part 2) of a screen example of the presenter selection screen. In FIG. 20, a presenter selection screen 2000 is a screen on which a presenter of an opinion on a task K having a low degree of difficulty can be selected. On the presenter selection screen 2000, gaze images 2001 and 2002 are displayed.

  The gaze image 2001 is an image of the student S2 (Yamada) whose gaze rate in the presenter selection mode in the current lecture is 80% or more. The gaze image 2002 is an image of the student S8 (Asai) whose gaze rate in the presenter selection mode in the current lecture is 80% or more.

  Further, on the presenter selection screen 2000, by clicking any one of the gaze images 2001 and 2002, the presenter of the opinion on the subject K can be selected.

  According to the presenter selection screen 2000, the teacher can determine, from the gaze images 2001 and 2002, the students S2 and S8 who are more likely to face themselves during the presenter selection mode. For this reason, the teacher can determine that the students S2 and S8 are considerably confident students. Further, since the difficulty of the assignment K is low, the teacher can easily select the students S2 and S8 as the presenters even if the students S2 and S8 are not good at the subject corresponding to the assignment K. Thereby, even a student who is not good at the subject corresponding to the assignment K can gain confidence by answering the assignment K with low difficulty.

(Education support processing procedure of teacher terminal 201)
Next, the educational support processing procedure of the teacher terminal 201 will be described.

  FIG. 21 is a flowchart illustrating an example of the educational support processing procedure of the teacher terminal 201. In the flowchart of FIG. 21, first, the teacher terminal 201 executes an opinion collection process (step S2101). The specific processing procedure of the opinion collection processing will be described later with reference to FIG.

  Next, the teacher terminal 201 executes the in-lecture process (step S2102). The specific processing procedure of the in-lecture processing will be described later with reference to FIG. Then, the teacher terminal 201 executes the during-presentation process (step S2103), and ends the series of processes according to this flowchart. The specific processing procedure of the during-presentation processing will be described later with reference to FIGS.

<Specific processing procedure of opinion collection processing>
Next, a specific processing procedure of the opinion collection processing in step S2101 shown in FIG. 21 will be described.

  FIG. 22 is a flowchart illustrating an example of a specific processing procedure of the opinion collection processing. In the flowchart of FIG. 22, first, the teacher terminal 201 displays an assignment selection screen 1100 on the display 406 (step S2201).

  Then, the teacher terminal 201 determines whether or not the selection of the assignment ID of the assignment K to be examined by the student has been received (step S2202). Here, the teacher terminal 201 waits to accept the selection of the assignment ID (Step S2202: No).

  Then, when the selection of the assignment ID is received (Step S2202: Yes), the teacher terminal 201 extracts the assignment information corresponding to the selected assignment ID from the assignment DB 230 (Step S2203). Next, the teacher terminal 201 transmits the extracted assignment information to the IP addresses of the student terminals T1 to Tn in response to the assignment transmission button 1104 of the assignment selection screen 1100 being clicked (step S2204).

  Then, the teacher terminal 201 starts measuring the answer limit time of the assignment K (step S2205). Next, the teacher terminal 201 determines whether or not the response result for the assignment K has been received from the student terminal Ti (step S2206).

  Here, when the answer result is not received (step S2206: No), the teacher terminal 201 shifts to step S2210. On the other hand, when the response result is received (Step S2206: Yes), the teacher terminal 201 records the received response result in the response DB 240 (Step S2207).

  Then, the teacher terminal 201 refers to the answer DB 240 and generates screen information of the teacher matrix screen 1300, thereby displaying the teacher matrix screen 1300 on the display 406 (step S2208). Next, the teacher terminal 201 determines whether or not the response results of all the students have been received (step S2209).

  Here, when the response results of all the students have been received (step S2209: Yes), the teacher terminal 201 proceeds to step S2211. On the other hand, when the answer results of all the students have not been received (step S2209: No), the teacher terminal 201 determines whether or not the answer limit time of the assignment K has elapsed (step S2210).

  Here, when the answer time limit of the assignment K has not elapsed (step S2210: No), the teacher terminal 201 returns to step S2206. On the other hand, when the answer limit time of the assignment K has elapsed (step S2210: Yes), the teacher terminal 201 refers to the answer DB 240 and generates the screen information of the student matrix screen 1400, so that the student The display matrix screen 1400 is displayed (step S2211), and the process returns to the step where the opinion collection process is called. Thereby, the answer result of the student Si with respect to the assignment K can be collected.

<Specific processing procedure of processing during lecture>
Next, a specific processing procedure of the in-lecture processing in step S2102 shown in FIG. 21 will be described.

  FIG. 23 is a flowchart illustrating an example of a specific processing procedure of the in-lecture processing. In the flowchart of FIG. 23, first, the teacher terminal 201 sets the status flag to “1” (step S2301). Next, the teacher terminal 201 performs a flagging process (step S2302). The specific processing procedure of the flagging processing will be described later with reference to FIGS. 27 and 28.

  Then, the teacher terminal 201 determines whether or not an instruction to shift to the presenter selection mode has been received (step S2303). Here, when the instruction to shift to the presenter selection mode has not been received (step S2303: No), the teacher terminal 201 returns to step S2302. On the other hand, when the instruction to shift to the presenter selection mode is received (step S2303: Yes), the teacher terminal 201 returns to the step in which the in-lecture process is called.

  As a result, the flagging process can be repeatedly executed until an instruction to shift to the presenter selection mode is received.

<Specific processing procedure during presentation>
Next, a specific processing procedure of the during-presentation processing in step S2103 shown in FIG. 21 will be described.

  24 to 26 are flowcharts illustrating an example of a specific processing procedure of the during-presentation processing. In the flowchart of FIG. 24, first, the teacher terminal 201 sets “i” of the student Si to “i = 1” (step S2401), sets the number of students to n, and selects the student Si from the students S1 to Sn ( Step S2402).

  Next, the teacher terminal 201 refers to the gaze / depression flag table 250 to determine the number of records R2 of the status flag “1” and the number of depression flags F3 of the depression flag “1” of the student Si of the status flag “1”. The counting is performed (step S2403). Then, the teacher terminal 201 divides the counted number of depression flags F3 by the number of records R2 to calculate the depression rate of the student Si in the current lecture (step S2404).

  Next, the teacher terminal 201 refers to the gaze / depression flag table 250 to determine the number R2 of records of the state flag “1” and the number F4 of gaze flags “1” of the student Si of the state flag “1”. The counting is performed (step S2405). Then, the teacher terminal 201 calculates the gaze rate of the student Si in the current lecture by dividing the counted number of gaze flags F4 by the number of records R2 (step S2406).

  Then, the teacher terminal 201 updates the current lecture column of the second gaze / depression time record DB 270 (step S2407). Next, the teacher terminal 201 increments “i” of the student Si (step S2408), and determines whether “i” has become larger than “n” (step S2409).

  Here, when “i” is equal to or less than “n” (step S2409: No), the teacher terminal 201 returns to step S2402. On the other hand, when “i” is larger than “n” (step S2409: Yes), the teacher terminal 201 sets the status flag to “2” (step S2410).

  Then, the teacher terminal 201 performs a flagging process (step S2411), and proceeds to step S2501 shown in FIG. The specific processing procedure of the flagging processing will be described later with reference to FIGS. 27 and 28.

  In the flowchart of FIG. 25, first, the teacher terminal 201 sets “i” of the student Si to “i = 1” (step S2501), sets the number of students to n, and selects the student Si from the students S1 to Sn (step S2501). Step S2502). Next, the teacher terminal 201 acquires the gaze rate of the student Si in the past lecture accumulation column and the gaze ratio of the student Si in the current lecture column with reference to the first gaze / depression time record DB 260 (step). S2503).

  Then, the teacher terminal 201 determines whether or not the difference (increase in gaze rate) obtained by subtracting the gaze rate of the student Si in the past lecture accumulation section from the gaze rate of the student Si in the current lecture section is equal to or larger than the threshold α. (Step S2504). Here, when the difference (increase in the gaze rate) is equal to or more than the threshold α (step S2504: Yes), the teacher terminal 201 displays the gaze image of the student Si in the area corresponding to the student Si on the teacher matrix screen 1300. (Step S2505).

  Next, the teacher terminal 201 increments “i” of the student Si (step S2506), and determines whether or not “i” is larger than “n” (step S2507). Here, when “i” is equal to or smaller than “n” (Step S2507: No), the teacher terminal 201 returns to Step S2502. On the other hand, if “i” is larger than “n” (step S2507: YES), the procedure moves to step S2601 shown in FIG.

  When the difference (increase in the gaze rate) is less than the threshold value α in step S2504 (step S2504: No), the teacher terminal 201 refers to the first gaze / depression time record DB 260 and stores the past lecture accumulation column. Of the student Si and the current rate of the student Si in the lecture section are obtained (step S2508).

  Then, the teacher terminal 201 determines whether or not the difference (the increase in the depression rate) of the student Si in the past lecture accumulation column minus the depression rate of the student Si in the current lecture column is equal to or greater than the threshold α. (Step S2509). Here, when the difference (increase in the downward ratio) is less than the threshold value α (step S2509: No), the teacher terminal 201 proceeds to step S2506.

  On the other hand, when the difference (increase in the depression rate) is equal to or larger than the threshold α (step S2509: Yes), the teacher terminal 201 displays the depression image of the student Si in the area corresponding to the student Si on the teacher matrix screen 1300. Then (step S2510), the process moves to step S2506.

  In the flowchart of FIG. 26, first, the teacher terminal 201 determines whether or not the presenter of the opinion on the assignment K has been selected (step S2601). Here, when the presenter has not been selected (step S2601: No), the teacher terminal 201 shifts to step S2411 shown in FIG.

  On the other hand, if the presenter is selected (step S2601: Yes), the teacher terminal 201 refers to the answer DB 240 and displays the screen information of the presentation screen 1500 that enlarges and displays the opinion of the student Si selected as the presenter. As a result, the presentation screen 1500 is displayed on the electronic blackboard 202 (step S2602).

  Next, the teacher terminal 201 sets the status flag to “3” (step S2603). Then, the teacher terminal 201 performs a flagging process (step S2604). The specific processing procedure of the flagging processing will be described later with reference to FIGS. 27 and 28.

  Next, the teacher terminal 201 determines whether a presentation end request has been received (step S2605). Here, when the presentation end request has not been received (step S2605: No), the teacher terminal 201 returns to step S2604. On the other hand, when the presentation end request is received (Step S2605: Yes), the teacher terminal 201 displays the student matrix screen 1400 on the electronic blackboard 202 (Step S2606).

  Next, the teacher terminal 201 determines whether a lesson end request has been received (step S2607). Here, when the lesson end request has not been received (step S2607: No), the teacher terminal 201 proceeds to step S2410 shown in FIG.

  On the other hand, when the lesson end request is received (step S2607: Yes), the teacher terminal 201 updates the current lecture column of the first gaze / depression time record DB 260 (step S2608) and calls the process during presentation. Return to step. This allows the teacher matrix screen 1300 to display the gaze image or the gaze image of the student Si having a large increase in the gaze rate or the gaze rate, thereby supporting the selection of the presenter of the opinion on the assignment K.

<Specific processing procedure of flagging processing>
Next, a specific processing procedure of the flagging processing of step S2302 shown in FIG. 23, step S2411 shown in FIG. 24, and step S2604 shown in FIG. 26 will be described.

  FIGS. 27 and 28 are flowcharts illustrating an example of a specific processing procedure of the flagging processing. In the flowchart of FIG. 27, first, the teacher terminal 201 acquires the latest recording date and time from the gaze / depression flag table 250 (step S2701). Next, the teacher terminal 201 determines whether or not a predetermined time T1 has elapsed from the acquired latest recording date and time (step S2702). The predetermined time T1 is, for example, 0.1 [s].

  Here, if the predetermined time T1 has not elapsed from the latest recording date and time (step S2702: No), the teacher terminal 201 returns to the step that called the flagging process. On the other hand, when the predetermined time T1 has elapsed from the latest recording date and time (Step S2702: Yes), the teacher terminal 201 acquires the latest captured image captured by the camera 300 (Step S2703).

  Next, the teacher terminal 201 sets “i” of the student Si to “i = 1” (step S2704), sets the number of students to n, and selects the student Si from the students S1 to Sn (step S2705). Then, the teacher terminal 201 performs a face recognition process on the image of the student Si (step S2706). A specific processing procedure of the face recognition processing will be described later with reference to FIG.

  Next, the teacher terminal 201 increments “i” of the student Si (step S2707), and determines whether or not “i” is larger than “n” (step S2708). Here, when “i” is equal to or less than “n” (step S2708: No), the teacher terminal 201 returns to step S2705.

  On the other hand, when “i” is larger than “n” (step S2708: Yes), the teacher terminal 201 determines whether or not the current state flag is “2” (step S2709). Here, if the status flag is not “2” (step S2709: No), the teacher terminal 201 returns to the step that called the flagging process.

  On the other hand, if the status flag is “2” (step S2709: Yes), the teacher terminal 201 acquires the current date and time of updating the lecture section from the first gaze / depression time record DB 260 (step S2710). Then, the teacher terminal 201 determines whether a predetermined time T2 has elapsed from the acquired update date and time (step S2711). The predetermined time T2 is, for example, 2 [s]. Note that the predetermined time T1 is a time representing an interval for acquiring and recognizing an image, and the predetermined time T2 is a time representing an interval for calculating a look-down rate and a gaze rate using a result of image recognition. The look-down ratio and the gaze ratio may be calculated every time an image is acquired, but the predetermined time T2 is set because the efficiency is reduced.

  Here, when the predetermined time T2 has not elapsed from the update date and time (step S2711: No), the teacher terminal 201 returns to the step that called the flagging process. On the other hand, when the predetermined time T2 has elapsed from the update date and time (step S2711: Yes), the teacher terminal 201 shifts to step S2801 shown in FIG.

  In the flowchart of FIG. 28, first, the teacher terminal 201 sets “i” of the student Si to “i = 1” (step S2801), sets the number of students to n, and selects the student Si from the students S1 to Sn (step S2801). Step S2802).

  Next, the teacher terminal 201 refers to the gaze / depression flag table 250 to determine the number of records R1 of the status flag “2” and the number of depression flags F1 of the depression flag “1” of the student Si of the status flag “2”. It is counted (step S2803). Then, the teacher terminal 201 divides the counted number of depression flags F1 by the number of records R1 to calculate the depression rate of the student Si in the presenter selection mode in the current lecture (step S2804).

  Next, the teacher terminal 201 refers to the gaze / depression flag table 250 to determine the number R1 of records of the status flag “2” and the number F2 of gaze flags “1” of the student Si of the status flag “2”. The counting is performed (step S2805). Then, the teacher terminal 201 calculates the gaze rate of the student Si in the presenter selection mode in the present lecture by dividing the counted number of gaze flags F2 by the number of records R1 (step S2806).

  Then, the teacher terminal 201 updates the current lecture column of the first gaze / depression time record DB 260 (step S2807). Next, the teacher terminal 201 increments “i” of the student Si (step S2808), and determines whether “i” has become larger than “n” (step S2809).

  Here, when “i” is equal to or less than “n” (step S2809: No), the teacher terminal 201 returns to step S2802. On the other hand, when “i” is larger than “n” (step S2809: Yes), the teacher terminal 201 returns to the step of calling the flagging process.

  Thereby, the current lecture column of the first gaze / depression time record DB 260 can be updated periodically (every predetermined time T2).

<Specific processing procedure of face recognition processing>
Next, a specific processing procedure of the face recognition processing in step S2706 shown in FIG. 27 will be described.

  FIG. 29 is a flowchart illustrating an example of a specific processing procedure of the face recognition processing. In the flowchart of FIG. 29, first, the teacher terminal 201 acquires the area coordinates of the student Si from the student DB 220 (step S2901). Next, the teacher terminal 201 extracts an image of the face area specified from the area coordinates of the student Si from the captured image acquired in step S2703 shown in FIG. 27 (step S2902).

  Then, the teacher terminal 201 performs face recognition on the image of the extracted face region using the learning data for face recognition of the student Si (step S2903). Next, the teacher terminal 201 determines whether or not the student Si is facing down, based on the result of the face recognition (step S2904). Judgment of depression can be made by pattern recognition such that a part (eye, nose, mouth, etc.) which is a feature of the face cannot be recognized by image recognition, and there are many parts considered to be the head.

  Here, when the student Si is facing down (step S2904: Yes), the teacher terminal 201 sets the gaze flag “0” and the look-down flag “1” of the student Si in association with the recording date and time and the status flag. The information is recorded in the depression flag table 250 (step S2905), and the process returns to the step in which the face recognition process is called.

  On the other hand, when the student Si is not facing down (step S2904: No), the teacher terminal 201 detects the gaze of the student Si based on the image of the eyes of the student Si obtained by face recognition (step S2906). Then, the teacher terminal 201 determines whether or not the detected line of sight of the student Si faces the direction of the teacher (step S2907).

  Here, when the line of sight of the student Si is facing the direction of the teacher (step S2907: Yes), the teacher terminal 201 associates the recording date and the state flag with the gaze flag “1” of the student Si and the depression flag “ "0" is recorded in the gaze / depression flag table 250 (step S2908), and the process returns to the step in which the face recognition process is called.

  On the other hand, when the line of sight of the student Si is not facing the direction of the teacher (step S2907: No), the teacher terminal 201 associates the recording date and the state flag with the gaze flag “0” of the student Si and the downward flag “0”. Is recorded in the gaze / depression flag table 250 (step S2909), and the process returns to the step in which the face recognition process is called.

  Thereby, it is possible to record the gaze flag and the depression flag of the student Si.

(Procedure input processing procedure of student terminal Ti)
Next, the opinion input processing procedure of the student terminal Ti will be described.

  FIG. 30 is a flowchart illustrating an example of an opinion input processing procedure of the student terminal Ti. In the flowchart of FIG. 30, first, the student terminal Ti determines whether or not the assignment information of the assignment K has been received from the teacher terminal 201 (step S3001). Here, the student terminal Ti waits to receive the assignment information on the assignment K (Step S3001: No).

  Then, when receiving the assignment information of the assignment K (Step S3001: Yes), the student terminal Ti displays the answer screen 1200 on the display 406 (Step S3002). Next, the student terminal Ti determines whether or not a request to transmit an opinion content for the assignment K described on the answer screen 1200 has been received (step S3003).

  Here, the student terminal Ti waits to receive a transmission request of the opinion contents for the assignment K (step S3003: No). When the student terminal Ti receives the request to transmit the opinion contents for the assignment K (step S3003: Yes), the student terminal Ti transmits an answer result including the input opinion contents to the teacher terminal 201 (step S3004), and the flowchart shown in FIG. Is completed.

  Thereby, the content of the opinion of the student Si on the assignment K given by the teacher can be transmitted to the teacher terminal 201.

  As described above, according to the teacher terminal 201 according to the embodiment, based on the recognition result obtained by performing face recognition on the image of the face area of the student Si taken after receiving the instruction to shift to the presenter selection mode. Thus, the depression rate of the student Si in the presenter selection mode in the present lecture can be calculated. Then, according to the teacher terminal 201, the calculated depression rate of the student Si in the presenter selection mode in the current lecture can be output in association with the student Si.

  This makes it possible to present to the teacher the depression rate, which is an index for determining whether or not the student Si has confidence in the opinion on the assignment K, and can assist the teacher in selecting a presenter. For example, the teacher can judge that the higher the downward rate is, the less confident the student is.

  Further, according to the teacher terminal 201, it is possible to compare the look-down rate of the student Si in the presenter lecture in the presenter selection mode in the present lecture with the look-down rate in the past lecture. Then, according to the teacher terminal 201, as a result of the comparison, it is possible to output the identification information (for example, the student name and the downward image) of the student Si whose difference in the downward ratio is equal to or more than the threshold α.

  Thereby, the student Si who is more confident (or less confident) than usual about the opinion on the assignment K can be presented in a distinguishable manner. For example, the teacher can easily determine the student Si who wants to apply only this time, and it is difficult for the teacher to miss the timing at which the student wants to apply.

  Further, according to the teacher terminal 201, the gaze of the student Si is detected based on the image of the eyes of the student Si obtained by face recognition of the image of the student Si, and based on the detected result, the current lecture The gaze rate of the student Si in the presenter selection mode can be calculated. Then, according to the teacher terminal 201, the calculated gaze rate of the student Si in the presenter selection mode in the current lecture can be output in association with the student Si.

  Thus, the gaze rate, which is an index for determining whether or not the student Si is confident about the opinion on the assignment K, can be presented to the teacher, and it is possible to assist the teacher in selecting a presenter. For example, a teacher can determine that the higher the gaze rate is, the more confident the student is in his / her opinion.

  Further, according to the teacher terminal 201, it is possible to compare the gaze rate in the presenter's lecture mode in the present lecture with the gaze rate of the student Si in the presenter's selection mode in the current lecture. According to the teacher terminal 201, as a result of the comparison, it is possible to output identification information (for example, a student name and a gaze image) of the student Si whose difference in gaze rate is equal to or more than the threshold α.

  Thereby, the student Si who is more confident (or less confident) than usual about the opinion on the assignment K can be presented in a distinguishable manner. For example, the teacher can easily determine the student Si who wants to apply only this time, and it is difficult for the teacher to miss the timing at which the student wants to apply.

  Further, according to the teacher terminal 201, it is possible to compare the depression rate of the student Si in the present lecture with the depression rate of the student Si in the presenter selection mode in the present lecture. Then, according to the teacher terminal 201, as a result of the comparison, it is possible to output the identification information (for example, the student name and the downward image) of the student Si whose difference in the downward ratio is equal to or more than the threshold α.

  As a result, the student Si whose look-down ratio in the presenter selection mode in the current lecture is extremely high compared to during the current lecture, that is, suddenly falls down in order to not want to be regarded as a presenter. The student Si who is facing can be presented in a distinguishable manner.

  Further, according to the teacher terminal 201, the gaze rate of the student Si in the current lecture can be compared with the gaze rate of the student Si in the presenter selection mode in the current lecture. According to the teacher terminal 201, as a result of the comparison, it is possible to output identification information (for example, a student name and a gaze image) of the student Si whose difference in gaze rate is equal to or more than the threshold α.

  As a result, the student Si whose gaze rate in the presenter selection mode in the current lecture is extremely high compared to that in the current lecture, that is, the teacher suddenly needs to be assigned as the presenter. Can be presented in a distinguishable manner.

  Although specific recognition processing is disclosed in the above embodiments, the present invention is not limited to such embodiments, and general image recognition technology may be applied. For example, a recognition process based on a teacher image as used in facial expression recognition may be performed. As the teacher image, learning of the recognition process may be performed using the face gazing at the teacher and the face not gazing at the teacher as teacher data. Alternatively, learning of recognition processing may be performed using a confident expression and an unconfident expression as teacher data. Then, the identification information of the student may be output using a confident rate or a non-confident rate obtained from the recognition processing result.

  The presenter selection support method described in the present embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. The presenter selection support program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The presenter selection support program may be distributed via a network such as the Internet.

  Regarding the above-described embodiment, the following supplementary notes are further disclosed.

(Appendix 1)
Accepts an instruction to shift to the mode to select a presenter,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, a value indicating a ratio of the subject having the predetermined feature during the mode is calculated. And
Outputting the calculated value in association with the target person,
A presenter selection support program characterized by executing processing.

(Supplementary Note 2) The calculating process includes calculating a look-down rate indicating a rate at which the target person looks down during the mode based on the recognition result,
The output process outputs the calculated downward rate in association with the target person,
The presenter's selection support program according to supplementary note 1, characterized in that:

(Supplementary Note 3) In the computer,
With reference to a storage unit that stores the depression rate of the subject in the mode for selecting a presenter before the mode, the calculated depression rate and the depression of the subject stored in the storage unit are referred to. To perform further processing to compare with the rate,
The output processing includes:
As a result of the comparison, when a difference between the calculated depression rate and the depression rate of the subject stored in the storage unit is equal to or greater than a threshold value, identification information for identifying the subject is output. The presenter selection support program described in Appendix 2.

(Supplementary Note 4) In the computer,
Based on the image of the eye of the target person obtained by face recognition of the image of the target person, detecting the line of sight of the target person, further executing a process,
The calculating process calculates a gaze rate indicating a rate at which the target person is looking in a specific direction during the mode, based on the detected detection result,
The output process outputs the calculated gaze rate in association with the target person,
The presenter selection support program according to any one of supplementary notes 1 to 3, characterized in that:

(Supplementary note 5) The presenter selection support program according to supplementary note 4, wherein the specific direction is a direction in which a person who selects the presenter is located.

(Supplementary Note 6) In the computer,
Prior to the mode, with reference to the storage unit storing the gaze rate of the subject in the mode for selecting a presenter, the calculated gaze rate and the gaze of the subject stored in the storage unit To perform further processing to compare with the rate,
The output processing includes:
As a result of the comparison, when the difference between the calculated gaze rate and the gaze rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information for identifying the subject is output. Supplementary note 4 or 5 presenter selection support program.

(Appendix 7)
Accepts an instruction to shift to the mode to select a presenter,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, a value indicating a ratio of the subject having the predetermined feature during the mode is calculated. And
Outputting the calculated value in association with the target person,
A presenter selection support method characterized by performing a process.

(Supplementary Note 8) An instruction to shift to a mode for selecting a presenter is received,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, a value indicating a ratio of the subject having the predetermined feature during the mode is calculated. And
Outputting the calculated value in association with the target person,
An information processing device comprising a control unit.

Reference Signs List 101 Information processing device 110, 300 Camera 200 Education support system 201 Teacher terminal 202 Electronic blackboard 220 Student DB
230 Assignment DB
240 Answer DB
250 gaze / downturn flag table 260 first gaze / downtime record DB
270 Second Look and Look Time DB
1601 acquisition unit 1602 reception unit 1603 display control unit 1604 transmission unit 1605 reception unit 1606 recognition unit 1607 detection unit 1608 setting unit 1609 calculation unit 1610 update unit T1 to Tn student terminal

Claims (6)

On the computer,
Accepts an instruction to shift to the mode to select a presenter,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, the subject is in a state where the subject is facing down during the mode for each of the subjects. Calculate the depression rate indicating the percentage,
With reference to a storage unit that stores the depression rate of the subject in the mode for selecting a presenter before the mode, the calculated depression rate and the depression of the subject stored in the storage unit are referred to. Compare with the rate
As a result of the comparison, when the difference between the calculated depression rate and the depression rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information identifying the subject is output.
A presenter selection support program characterized by executing processing. On the computer,
Based on the image of the eye of the target person obtained by face recognition of the image of the target person, detecting the line of sight of the target person, further executing a process,
The calculating process, based on the detected detection result, for each of the subjects, calculating a gaze rate indicating a rate of the subject is in a state of looking in a specific direction during the mode,
The output process outputs the calculated gaze rate in association with the target person,
The presenter's selection support program according to claim 1, wherein: On the computer,
Prior to the mode, with reference to the storage unit storing the gaze rate of the subject in the mode for selecting a presenter, the calculated gaze rate and the gaze of the subject stored in the storage unit To perform further processing to compare with the rate,
The output processing includes:
As a result of the comparison, when the difference between the calculated gaze rate and the gaze rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information for identifying the subject is output. The presenter selection support program according to claim 2. Computer
Accepts an instruction to shift to the mode to select a presenter,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, the subject is in a state where the subject is facing down during the mode for each of the subjects. Calculate the depression rate indicating the percentage,
With reference to a storage unit that stores the depression rate of the subject in the mode for selecting a presenter before the mode, the calculated depression rate and the depression of the subject stored in the storage unit are referred to. Compare with the rate
As a result of the comparison, when the difference between the calculated depression rate and the depression rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information identifying the subject is output.
A presenter selection support method characterized by performing a process. Accepts an instruction to shift to the mode to select a presenter,
Based on a recognition result obtained by recognizing a predetermined feature from a face image of the subject taken after receiving the transfer instruction, the subject is in a state where the subject is facing down during the mode for each of the subjects. Calculate the depression rate indicating the percentage,
With reference to a storage unit that stores the depression rate of the subject in the mode for selecting a presenter before the mode, the calculated depression rate and the depression of the subject stored in the storage unit are referred to. Compare with the rate
As a result of the comparison, when the difference between the calculated depression rate and the depression rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information identifying the subject is output.
An information processing device comprising a control unit. On the computer,
Accepts an instruction to shift to the mode to select a presenter,
Based on the image of the eye of the subject obtained by face recognition of the face image of the subject taken after receiving the transfer instruction, the gaze of the subject is detected,
Based on the detected detection result, for each of the subjects, calculate a gaze rate indicating a rate that the subject is in a state of looking in a specific direction during the mode,
Prior to the mode, with reference to the storage unit storing the gaze rate of the subject in the mode for selecting a presenter, the calculated gaze rate and the gaze of the subject stored in the storage unit Compare with the rate
As a result of the comparison, when a difference between the calculated gaze rate and the gaze rate of the subject stored in the storage unit is equal to or greater than a threshold, the identification information for identifying the subject is output.
A presenter selection support program characterized by executing processing.

Images