JP2018060375A - Information processing system, information processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide student information needed by instructors according to lecture scenes.SOLUTION: An information processing system comprises a video camera 100 for capturing an image of students, an information processing device 200 for specifying a student based on the captured image to obtain information on a lecture of the specified student, and a terminal device 300 for obtaining information from the information processing device 200 to display the student information on a display device. The terminal device 300 switches between an overall information display screen for displaying information on a position of the student in a classroom and an information presentation screen for displaying information on the lecture of the student according to a position and an orientation of the instructor in the classroom, and then displays it on the display device.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing system, an information processing apparatus, and a program.

  The state of the student is analyzed using an image obtained by photographing the student of the lecture. Patent Document 1 discloses an extraction means for extracting face images of a plurality of students from an image obtained by photographing the attendance scenery of a plurality of students who take a class and the like based on the extracted face images. Determination means for determining a person's state, and an image generation means for generating a list image for displaying a list by adding an index indicating the state determined by the determination means to the face image of the student extracted by the extraction means An attendance state discriminating apparatus is disclosed.

JP 2006-330464 A

  When a lecturer is informed of student information in a lecture, information required by the lecturer may change depending on the scene of the lecture. There is a case where the information required by the lecturer cannot be provided only by analyzing the state of the student based on the image of the student.

  An object of the present invention is to make it possible to provide student information required by a lecturer according to the scene of a lecture.

An information processing system according to claim 1 of the present invention includes:
Image acquisition means for acquiring images of students in a lecture;
Identifying means for identifying a student based on the image obtained by the image obtaining means;
Information acquisition means for acquiring information on the lecture of the student specified by the specification means;
A first screen that displays information on the position of the student in the room where the lecture is held, and the information on the student including the information on the lecture acquired by the information acquisition unit regarding the student specified by the specifying unit; Display means for displaying a second screen for displaying information;
An operation means for performing an operation for causing the display means to switch display between the first screen and the second screen;
An information processing system comprising:
An information processing system according to claim 2 of the present invention includes:
The display means can further display a third screen that displays both the location information of the student and the information of the student,
2. The information processing system according to claim 1, wherein the operation unit further performs an operation of switching display between the first screen and the second screen and the third screen. 3. It is.
An information processing system according to claim 3 of the present invention includes:
A lecturer information acquisition means for acquiring information including a position in a room where the lecture is held with respect to the lecturer in the lecture,
The said display means switches and displays the said 1st screen and the said 2nd screen according to the information of the position of the said lecturer acquired by the said lecturer information acquisition means, The display means characterized by the above-mentioned. Alternatively, the information processing system according to claim 2.
An information processing system according to claim 4 of the present invention includes:
The display means displays the first screen when the instructor is at the podium position based on the instructor position information acquired by the instructor information acquiring means, and the instructor is within a certain range from the students. 4. The information processing system according to claim 3, wherein the second screen on which information of students within the range is displayed is displayed when the user is in the area.
An information processing system according to claim 5 of the present invention includes:
A lecturer information acquisition means for acquiring information including a position in a room where the lecture is held with respect to the lecturer in the lecture,
The display means displays the first screen or the third screen when the instructor is at the podium position based on the instructor position information acquired by the instructor information acquisition means. The information processing system according to claim 2.
An information processing system according to claim 6 of the present invention includes:
The lecturer information acquisition means further acquires information on the direction of the lecturer,
The display means displays the second screen by switching the display target students according to the position information and orientation information of the lecturer acquired by the lecturer information acquisition means. An information processing system according to claim 3.
An information processing system according to claim 7 of the present invention includes:
The display means includes position information and information about a lecture regarding a student who is located in a direction in which the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the lecturer information acquisition means. The information processing system according to claim 6, wherein the second screen on which the information of the student is displayed is displayed.
An information processing system according to claim 8 of the present invention includes:
The image acquisition means further acquires an image of a lecturer in the lecture,
The information processing system according to claim 3, wherein the lecturer information acquisition unit analyzes the image acquired by the image acquisition unit and acquires the information of the lecturer.
An information processing system according to claim 9 of the present invention includes:
The information processing system according to claim 1, wherein the specifying unit repeatedly executes a process of specifying a student at least at a predetermined time during a lecture.
An information processing system according to claim 10 of the present invention includes:
The information processing system according to claim 9, wherein the specifying unit repeatedly executes a process of specifying a student even for a student who has been specified once.
An information processing apparatus according to claim 11 of the present invention includes:
An information acquisition means for acquiring information identifying a student in the lecture, information on the lecture of the student, and information on the position of the lecturer in the room where the lecture is held;
A first screen that displays information on the position of the student in the room where the lecture is held, information that identifies the student based on the information acquired by the information acquisition means, and information related to the lecture of the student Display means for displaying a second screen for displaying
In the information processing apparatus, the display unit may switch and display the first screen and the second screen according to the position information of the lecturer acquired by the information acquisition unit. is there.
An information processing apparatus according to claim 12 of the present invention includes:
The display means can further display a third screen that displays both the position information of the student and the information of the student, and includes the first screen, the second screen, and the third screen. The information processing apparatus according to claim 11, wherein the display is switched and displayed.
An information processing apparatus according to claim 13 of the present invention provides:
The information acquired by the information acquisition means further includes information on the orientation of the lecturer,
The display means displays the second screen by switching the students to be displayed according to the position information and orientation information of the lecturer acquired by the information acquisition means. An information processing apparatus according to item 11 or claim 12.
An information processing apparatus according to claim 14 of the present invention includes:
The display means includes position information and lecture information regarding a student who is located in a direction in which the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the information acquisition means. The information processing apparatus according to claim 13, wherein the second screen on which student information is displayed is displayed.
A program according to claim 15 of the present invention is:
Computer
An information acquisition means for acquiring information identifying a student in the lecture, information on the lecture of the student, and information on the position of the lecturer in the room where the lecture is held;
A first screen that displays information on the position of the student in the room where the lecture is held, information that identifies the student based on the information acquired by the information acquisition means, and information related to the lecture of the student As a display control means for switching the second screen for displaying the information according to the position information of the lecturer acquired by the information acquisition means and displaying the information on the display device,
It is a program characterized by functioning.
An information processing apparatus according to claim 16 of the present invention includes:
As a function of the display control means, it is possible to further display a third screen that displays both the position information of the student and the information of the student, the first screen, the second screen, and the first screen. The program according to claim 15, wherein the computer realizes a function of switching and displaying the third screen on a display device.
A program according to claim 17 of the present invention includes:
The information acquired by the information acquisition means further includes information on the orientation of the lecturer,
As a function of the display control means, according to the position information and orientation information of the lecturer acquired by the information acquisition means, a function of switching the display target students on the second screen to the display device, The program according to claim 15 or 16, which is realized by the computer.
An information processing apparatus according to claim 18 of the present invention provides:
As a function of the display control means, the position information and the lecture information regarding the student who is located in the direction the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the information acquisition means. The program according to claim 17, wherein the computer is caused to realize a function of displaying the second screen on which the information of the student including the information is displayed.

According to the first aspect of the present invention, it is possible to obtain detailed information of each student required by the instructor as compared with the configuration in which the state of the student is analyzed based on the image of the student.
According to the invention of claim 2, detailed information of each student can be obtained together with information on the position of the student.
According to the third aspect of the present invention, detailed information on the student required by the instructor is obtained according to the position of the instructor as compared with the configuration in which the information of the instructor is output without taking into consideration the state of the instructor. be able to.
According to invention of Claim 4, the detailed information of the student who exists in a fixed range with respect to a lecturer can be obtained.
According to the invention of claim 5, when the lecturer is on the platform, detailed information on each student can be obtained together with information on the position of the student.
According to the invention of claim 6, the detailed information of the student that the instructor needs according to the position and orientation of the instructor, as compared with the configuration in which the information of the instructor is output without taking into consideration the state of the instructor. Can be obtained.
According to the seventh aspect of the present invention, detailed information of each student required by the instructor, which is assumed from the position and orientation of the instructor, can be obtained together with information on the position of the student.
According to invention of Claim 8, compared with the structure which analyzes a student's state based on a student's image, a lecturer's state is specified from a lecturer's image and the information which a lecturer needs can be obtained. it can.
According to the ninth aspect of the present invention, it is possible to obtain detailed information of the student including participation in the course during the course by repeatedly executing the process of identifying the student.
According to the tenth aspect of the present invention, it is possible to obtain detailed information of the student including leaving the course halfway by repeatedly executing the process of identifying the student.
According to the invention of claim 11, in comparison with the configuration in which the information of the student is output without taking into consideration the state of the instructor, the detailed information of the student required by the instructor is obtained according to the position of the instructor. be able to.
According to the invention of claim 12, detailed information of each student can be obtained together with information on the position of the student.
According to the invention of claim 13, the detailed information of the student that the instructor needs according to the position and orientation of the instructor as compared with the configuration in which the information of the instructor is output without taking into consideration the state of the instructor. Can be obtained.
According to the invention of claim 14, detailed information of each student required by the instructor, which is assumed from the position and orientation of the instructor, can be obtained together with information on the position of the student.
According to the invention of claim 15, the computer that executes the program of the present invention requires a lecturer according to the position of the lecturer, as compared with the configuration in which the information of the student is output without considering the state of the lecturer. You can get detailed information about the students.
According to the invention of claim 16, in the computer for executing the program of the present invention, detailed information of each student can be obtained together with information on the position of the student.
According to the invention of claim 17, in the computer that executes the program according to the present invention, the instructor according to the position and orientation of the instructor, in comparison with the configuration in which the information of the student is output without considering the state of the instructor. You can get detailed information of students who need.
According to the invention of claim 18, in the computer executing the program of the present invention, detailed information of each student required by the instructor, which is assumed from the position and orientation of the instructor, together with information on the position of the student Can be obtained.

It is a figure which shows the structural example of the non-linguistic information evaluation system to which this embodiment is applied. It is a figure which shows the hardware structural example of information processing apparatus. It is a figure which shows the function structure of information processing apparatus. It is a figure which shows the hardware structural example of a terminal device. It is a figure which shows the function structure of a terminal device. FIG. 6A is a diagram for explaining a technique for specifying a region related to a human body using inter-frame feature values, and FIG. 6A is a diagram showing a person sitting sideways on a chair in one frame of a moving image. FIG. 6B is a diagram showing a state where the same person has moved forward in another frame of the moving image. It is an example of the display output of the terminal device which acquired the information and evaluation result of the student from the information processing apparatus, and is a diagram illustrating an example of a first screen that displays the entire classroom information. It is an example of the display output of the terminal device which acquired the information and evaluation result of the student from the information processing apparatus, and is a diagram illustrating an example of a second screen presenting the student information. It is a figure which shows the example of arrangement | positioning of a video camera. It is a figure which shows the example of a whole information display screen.

<Configuration of non-linguistic information evaluation system to which this embodiment is applied>
FIG. 1 is a diagram illustrating a configuration example of a non-language information evaluation system to which the present embodiment is applied. As shown in FIG. 1, the non-linguistic information evaluation system 10 according to the present embodiment outputs a video camera 100 as a moving image acquisition device, an information processing device 200 as a moving image analysis device, and an analysis result by the information processing device 200. A terminal device 300 as an output device, a learning management server 400 storing learning management data, and a personal information database (DB) 500 are provided. The video camera 100, the information processing device 200, the terminal device 300, the learning management server 400, and the personal information DB 500 are connected via the network 20, respectively.

  The network 20 is not particularly limited as long as it can perform information communication among the video camera 100, the information processing device 200, the terminal device 300, the learning management server 400, and the personal information DB 500. For example, the Internet or a LAN (Local Area Network) is used. Etc. A communication line used for information communication may be wired or wireless. The network 20 that connects the video camera 100 and the information processing apparatus 200, and the network 20 that connects the information processing apparatus 200, the terminal device 300, the learning management server 400, and the personal information DB 500 may be a common network. However, it may be a different network. Although not particularly illustrated, the network 20 is appropriately provided with a relay device such as a gateway or a hub for connecting a network or a communication line.

  The non-linguistic information evaluation system 10 of this embodiment analyzes a moving image of a person who is an evaluation object or a person constituting the evaluation object, and extracts and extracts non-language information such as the movement of the person and facial expressions. Evaluate the evaluation target based on non-linguistic information. The non-linguistic information evaluation system 10 according to the present embodiment is an object such as an interview or an evaluation of a participant in a place or scene where many people gather, such as classes, lectures, events, entertainment facilities, and the like. Evaluate the state of individuals in situations where individuals are fixed. The evaluation target, the evaluation item, the evaluation content, and the like are set according to the application target and application scene of the non-linguistic information evaluation system 10. For example, the evaluation target may be an individual person or a set (group, team, etc.) of a plurality of persons. The evaluation item is set, for example, whether the person or the set of persons to be evaluated is concentrated on something, whether it is actively active, etc. The evaluation content is, for example, a degree suitable for such an evaluation item ( Degree) etc. are judged. Hereinafter, in this embodiment, the case where this system is used for evaluating students (participants) who take lectures (classes) in a classroom will be described as an example. This evaluation will be described later with a specific application example.

  In the system shown in FIG. 1, the video camera 100 is an example of an image acquisition unit, and is installed at a position and orientation where a student who is an application target of evaluation according to the present embodiment is captured in a photographing range, and photographs the student. In this embodiment, a student's moving image image | photographed with the video camera 100 is analyzed, and non-language information, such as operation | movement and a facial expression, is extracted. Therefore, the type and the number of installed video cameras 100 are set so that the movements and facial expressions of the students can be identified according to the configuration of the evaluation target (whether it is an individual or a group), the installation location, the range of the shooting range, etc. The For example, a telephoto camera is used for photographing an individual in a wide place, and a wide-angle camera is used for photographing a plurality of persons existing in a wide range. Moreover, in order to image various parts of the student's body, a plurality of cameras may be installed in various directions. Further, by capturing a wide range using a high-resolution camera, a plurality of images may be acquired, and the obtained image may be enlarged so that an individual image is an analysis target. In the present embodiment, the video camera 100 has a function of transmitting a captured moving image as digital data to the information processing apparatus 200 via the network 20.

  The learning management server 400 is a computer (server) that manages learning management data for lectures. The learning management data includes, for example, learning plan information for managing the progress of the lecture, information on the students in the lecture, learning progress and results for each student, and each student in the case of group work. Contains group information. The learning management server 400 may be composed of a single computer or a plurality of computers connected to the network 20. In the latter case, the function as the learning management server 400 is realized by distributed processing by a plurality of computers.

  The personal information DB 500 is a database server that stores personal information of students. The personal information stored in the personal information DB 500 includes identification information such as name and student ID number, image data for face authentication, information on lectures attended, and the like. The server constituting the personal information DB 500 may be constituted by a single computer or may be constituted by a plurality of computers connected to the network 20. In the latter case, student information is distributed and stored in a plurality of computers.

  The information processing apparatus 200 is an example of a behavior evaluation unit, and is a computer (server) that analyzes a moving image taken by the video camera 100 to extract and evaluate non-linguistic information about the student. The information processing apparatus 200 may be configured by a single computer or may be configured by a plurality of computers connected to the network 20. In the latter case, the function as the information processing apparatus 200 of the present embodiment to be described later is realized by distributed processing by a plurality of computers.

  FIG. 2 is a diagram illustrating a hardware configuration example of the information processing apparatus 200. As illustrated in FIG. 2, the information processing apparatus 200 includes a central processing unit (CPU) 201 that is a control unit and a calculation unit, a RAM 202 and a ROM 203, an external storage device 204, and a network interface 205. The CPU 201 performs various controls and arithmetic processes by executing programs stored in the ROM 203. The RAM 202 is used as a working memory in the control and arithmetic processing by the CPU 201. The ROM 203 stores various data used in programs executed by the CPU 201 and control. The external storage device 204 is realized by, for example, a magnetic disk device or a non-volatile semiconductor memory that can read and write data, and stores a program that is expanded in the RAM 202 and executed by the CPU 201 and a result of arithmetic processing by the CPU 201. The network interface 205 is connected to the network 20 and transmits / receives data to / from the video camera 100 and the terminal device 300. The configuration example illustrated in FIG. 2 is merely an example of a hardware configuration that implements the information processing apparatus 200 with a computer. The specific configuration of the information processing apparatus 200 is not limited to the configuration example illustrated in FIG. 2 as long as the functions described below can be realized.

  FIG. 3 is a diagram illustrating a functional configuration of the information processing apparatus 200. As illustrated in FIG. 3, the information processing apparatus 200 includes a moving image data acquisition unit 210, a face detection unit 211, a student identification unit 212, a learning information acquisition unit 213, a region identification unit 220, and a motion detection unit 230. A non-linguistic information extraction unit 240, a reaction evaluation unit 250, and an output unit 260.

  For example, in the computer shown in FIG. 2, the moving image data acquisition unit 210 is realized by the CPU 201 executing a program and controlling the network interface 205. The moving image data acquisition unit 210 receives moving image data from the video camera 100 via the network 20. The received moving image data is stored, for example, in the RAM 202 or the external storage device 204 shown in FIG.

  For example, in the computer shown in FIG. 2, the face detection unit 211 is realized by the CPU 201 executing a program. The face detection unit 211 analyzes the moving image acquired by the moving image data acquisition unit 210 and detects a portion where a face is reflected. As a method for detecting a face, for example, an existing identification method realized by a digital camera or the like may be used.

  The student specifying unit 212 is realized by the CPU 201 executing a program in the computer shown in FIG. 2, for example. The student identifying unit 212 connects to the personal information DB 500 via the network interface 205. Then, an authentication process is performed using the face image detected by the face detection unit 211 and the image data for face authentication stored in the personal information DB 500, and the student shown in the moving image is specified.

  The learning information acquisition unit 213 is realized by the CPU 201 executing a program in the computer illustrated in FIG. 2, for example. The learning information acquisition unit 213 is connected to the learning management server 400 via the network interface 205. Then, using the student information specified by the student specifying unit 212 and the learning management data stored in the learning management server 400, information (lecture information) regarding the specified lecture for each student is acquired. .

  For example, in the computer shown in FIG. 2, the area identification unit 220 is realized by the CPU 201 executing a program. The region identifying unit 220 analyzes the moving image acquired by the moving image data acquiring unit 210 and identifies a region in which the part of the student extracted as non-linguistic information by the non-linguistic information extracting unit 240 in the subsequent stage is shown. Specifically, areas where the human body (the whole) is shown, areas where the human head, body, arms, hands, fingers, etc. are reflected, head face, eyes, mouth, nose, ears, etc. The area where the body is reflected, the area where the upper body and the lower body are reflected, and the area where each body feature point is reflected, etc. (Hereinafter, the whole body part or part of the human body is not particularly distinguished. Called). As the parts to be identified, all the predetermined parts may be identified, or parts used for these processes based on the extraction by the subsequent non-linguistic information extraction unit 240 and the evaluation by the reaction evaluation unit 250 Only may be identified.

  For example, in the computer shown in FIG. 2, the operation detection unit 230 is realized by the CPU 201 executing a program. The motion detection unit 230 identifies the body part shown in each region based on the identification result of the region identification unit 220, and detects the motion for each identified region. Specifically, motions such as head movements, face orientations, face component movements (eyes, mouth, etc.), arm and leg movements, body orientations, body movements (walking, etc.) are detected. . As the motion of the detection target, all of the predetermined motions regarding a predetermined portion may be detected as targets, or the content of the evaluation by the non-linguistic information extraction unit 240 in the subsequent stage or the evaluation by the reaction evaluation unit 250 may be used. Based on this, only the movement of the part used for these processes may be detected.

  The non-linguistic information extraction unit 240 is realized by the CPU 201 executing a program in, for example, the computer shown in FIG. The non-linguistic information extraction unit 240 extracts, based on the movement of the part detected by the motion detection unit 230, what is used for evaluation for each evaluation item of the reaction evaluation unit 250 (non-linguistic information) among the behaviors of the students. To do. In other words, the non-linguistic information extraction unit 240 is an action extraction unit that extracts an action defined as non-linguistic information issued by a student. Specifically, for example, nodding, moving the face in a specific direction or changing the direction of the face, changing facial expressions, moving the mouth, speaking, yawning, or falling asleep It extracts motions when you are in motion, moving your eyes, raising your hands, writing, typing your keyboard, turning, turning poor, etc.

  For example, in the computer shown in FIG. 2, the output unit 260 is realized by the CPU 201 executing a program and controlling the network interface 205. The output unit 260 transmits information on the evaluation result by the reaction evaluation unit 250 to the terminal device 300 via the network 20.

  The terminal device 300 is an example of an output unit, and is an information terminal (client) that outputs an evaluation result by the information processing device 200. As the terminal device 300, for example, a device including an image display unit as an output unit such as a personal computer, a tablet terminal, or a smartphone is used.

  FIG. 4 is a diagram illustrating a hardware configuration example of the terminal device 300. As illustrated in FIG. 4, the terminal device 300 includes a CPU 301, a RAM 302 and a ROM 303, a display device 304, an input device 305, and a network interface 306. The CPU 301 performs various controls and arithmetic processes by executing programs stored in the ROM 303. The RAM 302 is used as a work memory in the control and arithmetic processing by the CPU 301. The ROM 303 stores various data used in programs executed by the CPU 301 and control. The display device 304 is configured by a liquid crystal display, for example, and displays an image as a display unit under the control of the CPU 301. The input device 305 is realized by an input device such as a keyboard, a mouse, or a touch sensor, and is operated by an operator as an operation unit. As an example, when the terminal device 300 is a tablet terminal, a smartphone, or the like, a touch panel in which a liquid crystal display and a touch sensor are combined functions as a display unit (display device 304) and an operation unit (input device 305). The network interface 306 is connected to the network 20 and transmits / receives data to / from the video camera 100 and the terminal device 300. The configuration example illustrated in FIG. 4 is merely an example of a hardware configuration that implements the terminal device 300 with a computer. The specific configuration of the terminal device 300 is not limited to the configuration example illustrated in FIG. 4 as long as the functions described below can be realized.

  FIG. 5 is a diagram illustrating a functional configuration of the terminal device 300. As illustrated in FIG. 5, the terminal device 300 according to the present embodiment includes a processing result acquisition unit 310, a display image generation unit 320, a display control unit 330, and an operation reception unit 340.

  For example, in the computer shown in FIG. 4, the processing result acquisition unit 310 is realized by the CPU 301 executing a program and controlling the network interface 306. The processing result acquisition unit 310 is an example of an information acquisition unit, and receives student information (position information and lecture information) and evaluation result data from the information processing apparatus 200 via the network 20. As will be described later, the processing result acquisition unit 310 receives data indicating the position and orientation information of the lecturer extracted from the image captured by the video camera 100. The received data is stored, for example, in the RAM 302 in FIG.

  For example, in the computer shown in FIG. 4, the display image generating unit 320 is realized by the CPU 301 executing a program. The display image generation unit 320 generates an output image indicating the evaluation result based on the evaluation result data acquired by the processing result acquisition unit 310. The configuration and display mode of the generated output image can be set according to the evaluation items, evaluation contents, and the like. Details of the output image will be described later.

  For example, in the computer shown in FIG. 4, the display control unit 330 is realized by the CPU 301 executing a program. The display control unit 330 displays the output image generated by the display image generation unit 320 on, for example, the display device 304 in the computer shown in FIG. In addition, the display control unit 330 receives a command related to display on the display device 304 and performs control such as display switching based on the received command.

  For example, in the computer shown in FIG. 4, the operation receiving unit 340 is realized by the CPU 301 executing a program. The operation receiving unit 340 receives an input operation performed by the operator using the input device 305. Then, in accordance with the operation received by the operation receiving unit 340, the display control unit 330 performs display control of an output image or the like to the display device 304.

<Processing of area identification unit>
Processing performed by the area identification unit 220 of the information processing apparatus 200 will be described. The area identifying unit 220 identifies a part related to the motion of a person shown in the moving image from the moving image captured by the video camera 100. Various existing image analysis techniques may be applied to this part identification. For example, an existing identification method realized by a digital camera or the like may be used to identify a face or a smile. Moreover, based on the part (area | region) of the specific shape currently reflected on the moving image, arrangement | positioning of such a some part, etc., the area | region where the body part is reflected can be specified. Further, as an example, identification based on inter-frame feature values may be performed. Specifically, an inter-frame feature value is obtained based on a difference between two or more consecutive frames of moving image data. Here, as the interframe feature amount, for example, a color boundary (edge), a color change amount, a moving direction or a moving amount of an area specified by these, and the like are used. The inter-frame feature quantity for a preset time is accumulated, and the inter-frame feature quantity is classified and integrated based on the distance and similarity of the inter-frame feature quantity for each frame. Thereby, the area | region which changes in cooperation in a moving image is specified, and the area | region where the part of the body is reflected is identified.

  FIG. 6 is a diagram for explaining a method for specifying a region related to a human body using inter-frame feature values. FIG. 6A shows a person sitting sideways in one frame of the video, and FIG. 6B shows that the same person has moved forward in another frame of the video. It shows a state. In the example illustrated in FIG. 6, the region identification unit 220 identifies a range in which an approximate color is reflected based on the color boundary and the amount of change shown in FIG. Then, the region identification unit 220 compares the frame in FIG. 6A and the frame in FIG. 6B and surrounds the frame with a broken line frame in the image based on the moving direction and moving amount of the corresponding color range. In the area 221, it is recognized that a plurality of color ranges are moving in cooperation, and this area 221 is identified as an area in which the upper body of the human body is reflected. Referring to FIGS. 6A and 6B, the position and size of the region 221 change according to the movement of the color range constituting the human body (upper body). Although the two frames in FIGS. 6A and 6B are compared here, the human body is based on the result of accumulating inter-frame feature quantities such as a change in color range by comparing three or more frames. You may make it identify the area | region where is reflected.

<Processing of motion detection unit>
Processing by the motion detection unit 230 will be described. The motion detection unit 230 analyzes a region in which the body part identified by the region identification unit 220 is reflected, identifies which part is specifically reflected, and detects a motion for each identified part. Various existing image analysis techniques may be applied to this motion detection. The detected motion is a motion that can occur as a physical motion for each identified part. For example, moving your eyes closed or opening your mouth, changing your line of sight, moving your face up and down, left and right, bending your elbows and waving your arms, bending your fingers, stretching your hands and opening and closing your hands. Motion, bending and stretching of the waist, twisting of the body, bending and stretching of the knee, movement of shaking the legs, movement of the body by walking, etc. are detected. Note that these movements are merely examples, and movements that can be detected by the non-language information evaluation system 10 of the present embodiment are not limited to the movements presented above. In the present embodiment, the motion detection unit 230 may detect the movements of all parts identified as regions by the region identification unit 220 or specify the motions extracted by the non-linguistic information extraction unit 240 at the subsequent stage. Therefore, the detection may be limited to the movement for the purpose. For example, if only the nodding motion is extracted by the non-linguistic information extraction unit 240, it is only necessary to detect a head movement that changes the face direction up and down.

<Processing of non-linguistic information extraction unit>
Processing by the non-language information extraction unit 240 will be described. The non-linguistic information extraction unit 240 extracts meaningful actions that the student has consciously or unconsciously performed as non-linguistic information based on the movement of the part detected by the motion detection unit 230. For example, a motion of nodding is extracted from a motion of changing the face direction up and down, a motion of utterance or absence is extracted from a motion of moving the mouth, and a motion of raising a hand is extracted from a motion of raising an arm. Extraction of non-linguistic information is not performed based solely on the movement of the part detected by the motion detection unit 230. For example, the movement of the corresponding part before and after the detected movement, the surrounding part, or another person This is also performed in consideration of information such as movements, scenes where the movements are detected, and context (background). As a specific example, when a motion that continuously changes the orientation of the face up and down is performed within a specific time, this motion is extracted as a motion of nodding. On the other hand, when the face is directed upward, and after a certain amount of time has passed, it moves downward and returns to its original state, and this movement is extracted as an action looking up upward for thinking. Also, when the face is facing down and a certain amount of time has elapsed, this action is extracted as an action indicating that the person is dozing. In addition, these operation | movement and consideration information are only illustrations, and the operation | movement and consideration information which can be extracted as non-language information in the non-language information evaluation system 10 of this embodiment are not limited to the operation | movement and information which were shown above.

<Processing of reaction evaluation unit>
Processing by the reaction evaluation unit 250 will be described. The reaction evaluation unit 250 evaluates the response of the student according to the scene to which the non-language information evaluation system 10 is applied based on the non-language information extracted by the non-language information extraction unit 240. For example, when evaluating the responses of students in a lecture, the degree of concentration on the lecture is an evaluation item. In the case of a participatory class, the degree of concentration and aggressiveness of each student, the activity level of the entire class, and the like are evaluation items. Moreover, you may make it evaluate simply whether it is a positive (positive) reaction or a negative (negative) reaction with respect to the objective of the scene where the non-linguistic information evaluation system 10 is applied. The evaluation result can be determined in various forms according to the evaluation item, the purpose of the evaluation, and the like. For example, a binary evaluation such as whether or not he / she spoke during the class may be performed, or a multi-value that specifies the achievement level (evaluation level) of evaluation items such as concentration and aggressiveness in stages Evaluation may be performed. The reaction evaluation unit 250 may perform continuous evaluation over a certain period of time during which a lecture or lesson is performed, and generate time-series evaluation information that changes with the passage of time.

  When performing multivalued evaluation, according to the evaluation items evaluated in the response evaluation unit 250, the types of actions (hereinafter referred to as reaction actions) defined (extracted) as non-linguistic information used for the evaluation and The appearance mode of the reaction behavior is set. In other words, even if the reaction behavior is the same, the evaluation is different depending on the appearance mode. For example, the evaluation differs between a case where a specific reaction action extracted as non-linguistic information is performed once and a case where the specific reaction action is repeated a plurality of times or continued for a certain time or more.

  Furthermore, when performing multi-level evaluation, for example, the degree of evaluation may be specified based on the type of reaction behavior, the appearance frequency, the duration, and the like in the evaluation item. As an example, let us consider a case where a nodding action and a writing action are defined as reaction behaviors to be evaluated for evaluating the degree of concentration. The nodding operation represents a higher degree of concentration than the writing operation. In this case, when a nodding action appears as a type of reaction action, it is evaluated that the degree of concentration is higher (the degree of evaluation is higher) than when a writing action appears. Further, as the appearance frequency of the reaction behavior, it is evaluated that the degree of concentration is higher (the degree of evaluation is higher) than the case where there are many nodding motions within a certain time. Moreover, it is evaluated that the degree of concentration is higher (the degree of evaluation is higher) when the duration of the writing operation within a certain time is longer than when the duration is shorter. Note that these evaluation items and evaluation methods are merely examples, and the evaluation items and evaluation methods that can be taken by the non-linguistic information evaluation system 10 of the present embodiment are not limited to the evaluation items and evaluation methods presented above.

<Other examples of non-linguistic information evaluation>
As described above, in addition to performing detailed evaluation of reaction behavior as non-linguistic information based on the specific behavior of the student as an evaluation target, the activity evaluation of the student movement is simplified as an evaluation by the reaction evaluation unit 250 It is good also as a structure evaluated to. Specifically, for example, when the region detected by the motion detection unit 230 as the region representing the body of the student is moving, regardless of which motion of which part of the body represents the motion The activity is evaluated based only on the magnitude and speed of the movement. In addition, as will be described in detail later, when a group is composed of a plurality of students, if any of the students who make up the group is moving, the activity based on the movement is evaluated as the group activity, It may be evaluated that the more students are moving, the higher the activity level is.

<Example of display on terminal device>
FIG. 7 and FIG. 8 are diagrams illustrating examples of display output of the terminal device 300 that has acquired the student information and the evaluation result from the information processing device 200. FIG. 7 shows an example of a first screen (hereinafter referred to as an overall information display screen) that displays overall information of a classroom (room). In the example illustrated in FIG. 7, a screen schematically showing classroom seats is displayed on the display device 304 of the terminal device 300. In the illustrated screen, four seats are provided in four rows, and seat numbers 1 to 16 are assigned to each seat. In addition, a circle “◯” is displayed on the seat where the student is seated (in the example of FIG. 7, seat numbers 1 to 3, 6, 8, 9, and 11 to 16). Thereby, the position of the student is specified. In the example shown in FIG. 7, the arrangement of seats in the classroom is registered as learning plan information stored in the learning management server 400 as information on the classroom where the lecture is performed, and learning management is performed when the lecture is performed. What is necessary is just to acquire from the server 400. This information may be acquired via the information processing apparatus 200, or may be acquired by directly accessing the learning management server 400 from the terminal apparatus 300. The information on the seat where the student is seated specifies the seat when the student identification unit 212 of the information processing apparatus 200 identifies the student, and sends the information from the information processing apparatus 200 to the terminal device 300 as the student information. You may do it. In addition, a sensor that senses that a student is seated in the seat may be provided, and the terminal device 300 may acquire information directly from the sensor. When classroom information and seat information are acquired in the terminal device 300, the display image generation unit 320 generates a screen, and the display control unit 330 displays the generated screen on the display device 304 (see FIG. 5). In the example shown in FIG. 7, the overall information display screen is a screen schematically showing classroom seats, but the configuration of the overall information display screen is not limited to the illustrated example. For example, an image showing the entire classroom photographed by the video camera 100 is displayed on the display device 304 of the terminal device 300, and a seat number, a display indicating the presence or absence of seating, etc. are superimposed and displayed at the position of each seat. An overall information display screen may be configured.

  FIG. 8 is a diagram illustrating an example of a second screen (hereinafter, an information presentation screen) for presenting student information. For example, when any seat is selected on the screen shown in FIG. 7, the information presentation screen is generated and displayed as a screen including information on the students seated in the selected seat. The seat is selected, for example, when the operator operates the input device 305 (see FIG. 4) such as a touch panel to designate an arbitrary seat. When the operation reception unit 340 (see FIG. 5) of the terminal device 300 receives this operation, the display image generation unit 320 generates an information presentation screen based on the information of the students who are seated in the selected seat.

  In the example shown in FIG. 8, the information of the students who are seated in seat numbers 1 to 3 is displayed. For each student, the name, student ID number (No.), attendance rate, and grade are displayed. Of these pieces of information, the name and student ID number are obtained from the personal information DB 500, for example. Further, the attendance rate and the grade are obtained from the learning management server 400, for example. In the example shown in FIG. 8, the name of the student with seat number 3 is “No applicable person”, and all other information is not displayed. This means that the student identification unit 212 of the information processing apparatus 200 has failed to authenticate the student seated in the seat of seat number 3. In the example shown in FIG. 8, the information presentation screen is a screen showing information of several students arranged in order of seat numbers, but the configuration of the information presentation screen is not limited to the illustrated example. For example, a portion in which a seat with a corresponding seat number is captured from an image captured by the video camera 100 is extracted and displayed on the display device 304 of the terminal device 300, and the seat number and the trainee's The overall information display screen may be configured so as to superimpose a display or the like.

  In the present embodiment, the student specifying unit 212 of the information processing apparatus 200 repeatedly executes authentication processing for each student in the classroom regularly or at a predetermined timing. For example, during the lecture, the authentication process is repeatedly executed for all students regardless of whether or not the authentication is successful. Therefore, even if a student who is “no relevant person” like the student of seat number 3 shown in FIG. 8 is authenticated in the subsequent authentication process, each information is displayed. When the student identification unit 212 repeatedly executes the authentication process, even if there is a student who is late for the lecture, this can be detected. In addition, the student identification unit 212 repeatedly executes the authentication process even after the student is successfully authenticated. Thereby, when there is a student who has left during the lecture or a student who has moved his / her seat, this can be detected. It is not always necessary to repeat the authentication process throughout the course of the lecture. At least a predetermined time during the lecture (for example, a certain time from the start of the lecture and a certain time before the lecture ends) Etc.) may be set to be executed.

  In the present embodiment, since face authentication is performed using image data stored in the personal information DB 500, the attendance is displayed as “no applicable person” like the attendee of seat number 3 shown in FIG. The person means that a face image that can be face-authenticated by the video camera 100 has not been photographed, or that the person is not registered in the personal information DB 500. If a person who does not have the qualification to attend the lecture arrives at the seat and the person is a person registered in the personal information DB 500, it may be a person who does not have the qualification according to the authentication process of the student identification unit 212. Identified.

  In the above example, the student is selected from the entire information display screen and the information display screen is displayed. However, the student is automatically selected according to the positional relationship between the terminal device 300 and the student. The information presentation screen may be displayed. For example, the terminal device 300 is a portable device such as a tablet terminal. When the lecturer (operator of the terminal device 300) is on the podium (lecture), a screen showing information on the entire classroom is displayed. On the other hand, when the instructor walks around the classroom while holding the terminal device 300, the switching control is performed so that information on the student in the direction the instructor is facing is displayed on the terminal device 300 according to the position and orientation of the instructor. . The position and orientation of the lecturer are specified based on, for example, an image of the lecturer taken by the video camera 100. Since the instructor walks near the student, the instructor is also photographed by the video camera 100 for photographing the student. Therefore, it is not necessary to prepare the video camera 100 for photographing the lecturer.

  FIG. 9 is a diagram illustrating an example of the arrangement of the video camera 100. In the example shown in FIG. 9, several students are seated on each of several tables arranged in the classroom. In addition, a lecturer 30 stands on one side of the classroom. In the example shown in FIG. 9, as the video camera 100, two wide-area shooting cameras 100 a (first acquisition means) and one enlarged shooting camera 100 b (second acquisition means) are provided. One of the two wide area photographing cameras 100a is arranged so as to photograph the inside of the classroom from one side of the classroom (the person with the lecturer 30). The other is arranged so that the inside of the classroom is photographed from the opposite side of the classroom. By arranging in this way, even if the student's face is behind other students or a table when viewed from one wide area shooting camera 100a, the other wide area shooting camera 100a allows the student's face. A face image is acquired.

  The enlargement photographing camera 100b has a variable enlargement ratio and is installed on a movable frame that can change the photographing direction. For example, the enlargement ratio and the shooting direction are controlled by a control signal from an external device. In addition, a human sensor or the like may be used to automatically track the instructor 30 or to automatically control the trainees in order. In the example shown in the figure, one enlargement camera 100b is shown, but a plurality of enlargement cameras 100b may be provided. Further, the enlargement photographing camera 100b may be installed on a movable base so that the lecturer 30 and the student can be photographed from different positions. In this case, the movement of the magnifying camera 100b may be controlled by, for example, an external control signal, or may be configured to move autonomously over a predetermined trajectory over an appropriate time. By setting it as such a structure, it can reduce that the lecturer 30 or a specific student becomes a state which cannot be image | photographed.

  The motion detection unit 230 of the information processing device 200 identifies the motion of the lecturer among the detected motions. Unlike the student, the instructor performs a special operation such as walking around the student, so the instructor and the student may be identified based on such a special operation. Further, the motion detection unit 230 identifies the position and orientation of the lecturer in the classroom based on the identified behavior of the lecturer. That is, in this configuration, the motion detection unit 230 functions as a lecturer information acquisition unit that acquires information on a lecturer. The location information in the classroom may be registered in the learning management server 400, for example, and acquired from the learning management server 400 together with seat information and the like. When the position and orientation of the instructor are specified, the seat in the direction facing the instructor is specified from the seat information at a position close to the instructor (within a predetermined range). The information of the student who is seated in the specified seat is sent to the terminal device 300. The terminal device 300 generates and displays an information presentation screen based on the received student information.

  Further, even when the instructor gets off the podium and is near the student, the screen displayed on the display device 304 is changed from the information presentation screen to the entire information display screen by performing a specific operation in the terminal device 300. Switch to. For example, when the input device 305 of the terminal device 300 is a touch panel combined with the display device 304, a button object for screen switching is displayed at an appropriate position on the information presentation screen, and the instructor touches this button object. By doing so, the screen may be switched.

  Another example of display output by the terminal device 300 of the present embodiment will be described. On the overall information display screen as shown in FIG. 7, the evaluation result of the information processing apparatus 200 for the student or the group of students may be displayed. For example, when the image of the student identified in the moving image is actively moving, it is evaluated that the activity level is high, and the display device 304 of the terminal device 300 displays the activity level visually.

  FIG. 10 is a diagram illustrating an example of the entire information display screen. In the example shown in FIG. 10, unlike the example shown in FIG. 7, there are several groups of students as shown in FIG. 9. Discussions and activities shall be conducted. In the illustrated example, the students are divided into four groups (groups A to D), and each group is expressed so as to be visually recognized by drawing a bold line on the screen displayed on the display device 304. ing.

  In the illustrated example, it is assumed that the activity level of the student is evaluated by numerical values in five stages from value “1” to value “5” in order from the lowest activity level. The evaluation value of the activity level of each student is displayed on the screen displayed on the display device 304 so as to be superimposed on each student. The evaluation value of the activity level of each group is the total value of the evaluation values of the students belonging to the group, and is displayed along with the group name. In the illustrated example, the activity evaluation value of group A is “18”, the activity evaluation value of group B is “15”, the activity evaluation value of group C is “14”, and the activity value of group D is The evaluation value is “15”. In the example shown in the figure, the activity level of the students and the group is expressed by an evaluation value (numerical value). However, a color indicating the level of activity level is set in advance, and each student image shown in FIG. You may display the mark of the color corresponding to the activity level of the evaluated student, and you may display the frame line showing a group by the color corresponding to the activity level of the evaluated group.

  The instructor can quantitatively recognize the activity of each student or each group by referring to the screen configured as shown in FIG. 10 displayed on the display device 304 of the terminal device 300. It is also possible to observe what kind of discussions and activities are being conducted by paying attention to the group with high activity, or to go to the group with low activity and give advice to increase the activity. it can.

  In the example shown in FIG. 10, the students are grouped for each table. However, when there is no standard item such as a table, the grouping may be performed based on the actions of each student. For example, it is considered that the students in the same group often synchronize or synchronize their operations. Therefore, when the motion detection unit 230 of the information processing apparatus 200 analyzes the moving image acquired by the video camera 100, a group of areas corresponding to the human body that are moving synchronously or moving in synchronization is grouped. May be set. When grouping is performed in this way, erroneous grouping different from the actual group may be performed. In such a case, the operator of the terminal device 300 may correct the grouping thick line drawn on the screen of the display device 304 to perform correct grouping. When such an operation is performed, the corrected grouping information is sent from the terminal device 300 to the information processing device 200. In the information processing apparatus 200, when the correction information is received, the operation detection unit 230 re-specifies the group area based on the acquired correction information. Then, the reaction evaluation unit 250 evaluates the group activity based on the corrected grouping, and transmits the group activity to the terminal device 300 by the output unit 260.

  In the above embodiment, the overall information display screen for displaying the entire classroom information including information on the seat position (the position of the student), and the information presentation screen for displaying the student information including the lecture information are provided. The example of switching to the display device 304 of the terminal device 300 and displaying it has been described. However, the screen displayed on the display device 304 of the terminal device 300 is not limited to the above example. For example, the third screen may be configured to display the lecture information of each student superimposed on the overall information display screen, and simultaneously display the position information of each student and the lecture information.

<Other configuration examples>
The non-linguistic information evaluation system 10 according to the present embodiment has been described above, but the specific configuration of the present embodiment is not limited to the above. For example, in the above configuration, the information processing apparatus 200 processes a moving image acquired by the video camera 100, and the terminal apparatus 300 as an output unit displays and outputs the obtained evaluation result. In contrast, in the process of authenticating the student and displaying the student information on the terminal device 300, a still image may be used instead of a moving image. Further, the information processing apparatus 200 and the terminal apparatus 300 may be separated from each other, and the information processing apparatus 200 may also serve as an output unit. For example, the information processing apparatus 200 itself may be configured to include a display device such as a liquid crystal display, and display the evaluation result. In the above embodiment, the information processing apparatus 200 processes the image acquired by the video camera 100 to identify the student, acquires the lecture information of the identified student, and acquires the student (and group). We decided to evaluate the condition. That is, in the above-described embodiment, the information processing apparatus 200 functions as a specifying unit that specifies a student, an information acquisition unit that acquires lecture information for each student, and an evaluation unit that evaluates the state of the student. On the other hand, each function may be realized by a separate information processing apparatus. In the above embodiment, the student's image is acquired by shooting with the video camera 100, but the information processing apparatus 200 may analyze and evaluate separately prepared image data. For example, the image data separately taken and stored in a storage device such as a magnetic disk device may be read and evaluated. In the above embodiment, the entire information display screen displaying the entire information in the classroom only displays whether or not there are students in the seat (see the circle “◯” in FIG. 7). Information (for example, name, student ID number, etc.) that can identify each student may be displayed.

DESCRIPTION OF SYMBOLS 10 ... Non-language information evaluation system, 20 ... Network, 100 ... Video camera, 200 ... Information processing apparatus, 201 ... CPU, 202 ... RAM, 203 ... ROM, 204 ... External storage device, 205 ... Network interface, 210 ... Movie Data acquisition unit, 211 ... face detection unit, 212 ... student identification unit, 213 ... learning information acquisition unit, 220 ... region identification unit, 230 ... motion detection unit, 240 ... non-linguistic information extraction unit, 250 ... reaction evaluation unit, 260 ... output unit, 300 ... terminal device, 301 ... CPU, 302 ... RAM, 303 ... ROM, 304 ... display device, 305 ... input device, 306 ... network interface, 310 ... processing result acquisition unit, 320 ... display image generation Unit, 330 ... display control unit, 340 ... operation receiving unit

Claims (18)

Image acquisition means for acquiring images of students in a lecture;
Identifying means for identifying a student based on the image obtained by the image obtaining means;
Information acquisition means for acquiring information on the lecture of the student specified by the specification means;
A first screen that displays information on the position of the student in the room where the lecture is held, and the information on the student including the information on the lecture acquired by the information acquisition unit regarding the student specified by the specifying unit; Display means for displaying a second screen for displaying information;
An operation means for performing an operation for causing the display means to switch display between the first screen and the second screen;
An information processing system comprising: The display means can further display a third screen that displays both the location information of the student and the information of the student,
2. The information processing system according to claim 1, wherein the operation unit further performs an operation of switching display between the first screen and the second screen and the third screen. 3. . A lecturer information acquisition means for acquiring information including a position in a room where the lecture is held with respect to the lecturer in the lecture,
The said display means switches and displays the said 1st screen and the said 2nd screen according to the information of the position of the said lecturer acquired by the said lecturer information acquisition means, The display means characterized by the above-mentioned. Or the information processing system of Claim 2.   The display means displays the first screen when the instructor is at the podium position based on the instructor position information acquired by the instructor information acquiring means, and the instructor is within a certain range from the students. 4. The information processing system according to claim 3, wherein the second screen on which the information of students within the range is displayed is displayed when the user is in the area. A lecturer information acquisition means for acquiring information including a position in a room where the lecture is held with respect to the lecturer in the lecture,
The display means displays the first screen or the third screen when the instructor is at the podium position based on the instructor position information acquired by the instructor information acquisition means. The information processing system according to claim 2. The lecturer information acquisition means further acquires information on the direction of the lecturer,
The display means displays the second screen by switching the display target students according to the position information and orientation information of the lecturer acquired by the lecturer information acquisition means. The information processing system according to claim 3.   The display means includes position information and information about a lecture regarding a student who is located in a direction in which the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the lecturer information acquisition means. The information processing system according to claim 6, wherein the second screen on which the student information is displayed is displayed. The image acquisition means further acquires an image of a lecturer in the lecture,
The information processing system according to claim 3, wherein the lecturer information acquisition unit analyzes the image acquired by the image acquisition unit and acquires the information of the lecturer.   The information processing system according to claim 1, wherein the specifying unit repeatedly executes a process of specifying a student at least at a predetermined time during a lecture.   The information processing system according to claim 9, wherein the specifying unit repeatedly executes a process of specifying a student even for a student who has been specified once. An information acquisition means for acquiring information identifying a student in the lecture, information on the lecture of the student, and information on the position of the lecturer in the room where the lecture is held;
A first screen that displays information on the position of the student in the room where the lecture is held, information that identifies the student based on the information acquired by the information acquisition means, and information related to the lecture of the student Display means for displaying a second screen for displaying
The information processing apparatus, wherein the display means switches between the first screen and the second screen according to the position information of the lecturer acquired by the information acquisition means.   The display means can further display a third screen that displays both the position information of the student and the information of the student, and includes the first screen, the second screen, and the third screen. The information processing apparatus according to claim 11, wherein the display is switched and displayed. The information acquired by the information acquisition means further includes information on the orientation of the lecturer,
The display means displays the second screen by switching the students to be displayed according to the position information and orientation information of the lecturer acquired by the information acquisition means. Item 13. The information processing device according to item 11 or claim 12.   The display means includes position information and lecture information regarding a student who is located in a direction in which the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the information acquisition means. The information processing apparatus according to claim 13, wherein the second screen on which student information is displayed is displayed. Computer
An information acquisition means for acquiring information identifying a student in the lecture, information on the lecture of the student, and information on the position of the lecturer in the room where the lecture is held;
A first screen that displays information on the position of the student in the room where the lecture is held, information that identifies the student based on the information acquired by the information acquisition means, and information related to the lecture of the student As a display control means for switching the second screen for displaying the information according to the position information of the lecturer acquired by the information acquisition means and displaying the information on the display device,
A program characterized by functioning.   As a function of the display control means, it is possible to further display a third screen that displays both the position information of the student and the information of the student, the first screen, the second screen, and the first screen. The program according to claim 15, wherein the computer realizes a function of switching and displaying the screen of 3 on a display device. The information acquired by the information acquisition means further includes information on the orientation of the lecturer,
As a function of the display control means, according to the position information and orientation information of the lecturer acquired by the information acquisition means, a function of switching the display target students on the second screen to the display device, The program according to claim 15 or 16, wherein the program is implemented by the computer.   As a function of the display control means, the position information and the lecture information regarding the student who is located in the direction the lecturer is facing, according to the position information and orientation information of the lecturer acquired by the information acquisition means. The program according to claim 17, wherein the computer is caused to realize a function of displaying the second screen on which the information of the student including the information is displayed.

Images