CN112185195A - Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) - Google Patents
Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) Download PDFInfo
- Publication number
- CN112185195A CN112185195A CN202011173948.9A CN202011173948A CN112185195A CN 112185195 A CN112185195 A CN 112185195A CN 202011173948 A CN202011173948 A CN 202011173948A CN 112185195 A CN112185195 A CN 112185195A
- Authority
- CN
- China
- Prior art keywords
- terminal
- student
- video
- students
- teacher
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013473 artificial intelligence Methods 0.000 title description 17
- 230000003993 interaction Effects 0.000 claims abstract description 29
- 230000003542 behavioural effect Effects 0.000 claims abstract description 13
- 230000006399 behavior Effects 0.000 claims description 90
- 230000002452 interceptive effect Effects 0.000 claims description 7
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 9
- 230000008901 benefit Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 238000002372 labelling Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
- G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
Abstract
The embodiment of the invention provides a method for controlling a remote teaching classroom by an AI. The method comprises the following steps: connecting the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal; the teacher video picture shot by the first terminal is sent to at least one second terminal for playing, and the student video picture shot by the at least one second terminal is sent to the first terminal for playing; identifying behavioral characteristics in the student video pictures shot by the at least one second terminal; scoring the lecture listening behavior of the corresponding student based on the identified behavior characteristics; and sending feedback information to students whose lecture behavior scores exceed a preset threshold value. The method of the invention enables the lecturer and students in class to interact in real time in the remote classroom, and reduces the application cost. In addition, the embodiment of the invention provides a device for controlling the remote teaching classroom by the AI.
Description
Technical Field
The embodiment of the invention relates to the field of education, in particular to a method and a device for controlling an AI to control a remote teaching classroom.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Along with the progress of science and technology, the mode of teaching is no longer restricted to face-to-face lectures, in order to make more students can receive better education, a mode of giving lessons with remote teaching is popularized gradually, the remote teaching classroom can utilize the advantage of internet, utilize the mode of video signal to carry out the live broadcast, utilize characteristics such as directly perceived, quick, the on-the-spot sensation of internet is strong, the region is unrestricted, the audience can be divided, strengthen the learning effect of online study. In the prior art, the teaching of remote classroom is carried out, and the large-scale many people live classroom based on the internet is mainly, and the number of people toward going to class in this kind of live classroom is too much, and the teacher of giving a lecture communicates with the student through audio frequency and video to explain for the student through the mode of whiteboard. Although remote teaching can be realized in this way, the master teacher can only give lessons to students on the video side, but cannot observe the learning state of students on the video side who are listening to the lessons, and cannot interact with students on the video side, the remote teaching of the master teacher can be completed in this way, but the master teacher cannot interact with the students so that the master teacher cannot better know the learning conditions of the students who are listening to the lessons, and the learning efficiency of the students is greatly reduced due to poor autonomy of a plurality of students.
In addition, in the existing live classroom, generally, after a teacher initiates live broadcast in preset time, students click the link of the live broadcast classroom through a student terminal to attend classes. This approach does not guarantee that all students can access the live classroom simultaneously for class, possibly delaying or disturbing the teacher's progress of class due to different access times.
And thirdly, the current live classroom can not automatically sign in students participating in the live classroom, and can not automatically remind students not entering the live classroom.
Disclosure of Invention
In view of the defects that the present lecturer can not interact with students in class in real time, the learning results of the students in class are greatly reduced, the students cannot enter the class state in a unified way, and the students are automatically signed in and reminded of not being signed in to class, an improved method and device for controlling the remote teaching classroom by the AI are very needed to solve the above defects in the prior art.
In this context, embodiments of the present invention are intended to provide a method and apparatus for AI-controlled remote teaching class.
In a first aspect of embodiments of the present invention, there is provided a method for controlling a remote teaching classroom by an AI, comprising: connecting the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal; the teacher video picture shot by the first terminal is sent to at least one second terminal for playing, and the student video picture shot by the at least one second terminal is sent to the first terminal for playing; identifying behavioral characteristics in the student video pictures shot by the at least one second terminal; scoring the lecture listening behavior of the corresponding student based on the identified behavior characteristics; and sending feedback information to students whose lecture behavior scores exceed a preset threshold value.
In one embodiment of the invention, the first terminal is a teacher class terminal; the second terminal is a student class attending terminal.
In another embodiment of the invention, the method comprises: receiving a request which is initiated by a teacher through a first terminal and used for adding at least one second terminal needing audio-video interaction into a group; and initiating a call to all second terminals joined in the group.
In yet another embodiment of the invention, the method comprises: respectively connected with a first terminal and at least one second terminal; sending the video picture of the teacher shot by the first terminal to a server and forwarding the video picture to the at least one second terminal for playing through the server; and sending the video picture of the teacher shot by the at least one second terminal to a server and forwarding the video picture to the first terminal for playing through the server.
In yet another embodiment of the invention, the method comprises: segmenting the video of each student in the student video pictures shot by the at least one second terminal; and performing behavior model analysis on the segmented video of each student, and identifying the characteristic of trip.
In yet another embodiment of the invention, the method comprises: recognizing the face picture of each student according to the video pictures of each student after segmentation; comparing the face picture of each student with a student list; a reminder is sent to students who are not participating in the classroom.
In yet another embodiment of the invention, the method comprises: receiving an instruction of a teacher selecting at least one student through a first terminal; amplifying the video pictures of the selected students and displaying the video pictures on the first terminal; automatically muting other second terminals except the second terminal where the selected student is located; and displaying the interactive pictures of the teacher and the selected students on at least one second terminal in a split screen mode.
In a further embodiment of the invention, the behavioral characteristics comprise one or a combination of attention direction of the student, limb movement of the student, trunk movement of the student and duration of the movement.
In yet another embodiment of the invention, the method comprises: and classifying the behavior characteristics.
In yet another embodiment of the present invention, the categories of behavioral characteristics include: the device can not be used for listening to classes and interfering others.
In yet another embodiment of the invention, the method comprises: scoring the schroentgen lecture behavior based on the classification and duration of the behavior feature.
In yet another embodiment of the invention, the method comprises: and sending a prompt to the students whose lecture behavior scores exceed a preset threshold value through the personal terminals.
In a second aspect of embodiments of the present invention, there is provided an apparatus comprising: the connection module is used for connecting the first terminal to at least one second terminal so as to realize audio and video interaction between the first terminal and the at least one second terminal; the transmitting module is used for transmitting the teacher video picture shot by the first terminal to at least one second terminal for playing and transmitting the student video picture shot by the at least one second terminal to the first terminal for playing; the identification module is used for identifying the behavior characteristics in the student video pictures shot by the at least one second terminal; the scoring module is used for scoring the lecture attending behaviors of the corresponding students based on the identified behavior characteristics; and the feedback module is used for sending feedback information to students of which the class-attending behavior scores exceed a preset threshold value.
In another embodiment of the present invention, the first terminal is a teacher attendance terminal; the second terminal is a student class attending terminal.
In yet another embodiment of the invention, the apparatus comprises: the module is used for receiving a request which is initiated by a teacher through a first terminal and used for adding at least one second terminal needing audio-video interaction into a group; means for initiating a call to all second terminals joined in the group.
In yet another embodiment of the present invention, the apparatus comprises: a module for connecting with a first terminal and at least one second terminal, respectively; the module is used for sending the video picture of the teacher shot by the first terminal to a server and forwarding the video picture to the at least one second terminal for playing through the server; and the module is used for sending the video picture of the teacher shot by the at least one second terminal to a server and forwarding the video picture to the first terminal for playing through the server.
In yet another embodiment of the present invention, the apparatus comprises: a module for splitting the video of each student in the student video pictures shot by the at least one second terminal; and the module is used for performing behavior model analysis on the video of each student after segmentation and identifying the trip as a characteristic.
In yet another embodiment of the present invention, the apparatus comprises: a module for recognizing the face picture of each student according to the video pictures of each student after segmentation; a module for comparing the face picture of each student with a student list; a module for sending a reminder to students who are not participating in the classroom.
In yet another embodiment of the present invention, the apparatus comprises: the module is used for receiving an instruction of a teacher for selecting at least one student through the first terminal; a module for displaying the video pictures of the selected students on the first terminal in an enlarged manner; a module for automatically muting second terminals other than the second terminal where the selected student is located; and the module is used for displaying the interactive pictures of the teacher and the selected students on at least one second terminal in a split screen mode.
In a further embodiment of the invention, the behavioral characteristics comprise one or a combination of attention direction of the student, limb movement of the student, trunk movement of the student and duration of the movement.
In yet another embodiment of the present invention, the apparatus comprises: means for categorizing the behavioral characteristics.
In yet another embodiment of the present invention, the categories of behavioral characteristics include: the device can not be used for listening to classes and interfering others.
In yet another embodiment of the present invention, the apparatus comprises: a module for scoring the Schopper's lecture behavior based on the classification and duration of the behavior feature.
In yet another embodiment of the present invention, the apparatus comprises: and the module is used for sending a prompt to students with the class-attending behavior scores exceeding a preset threshold value through the personal terminal.
According to the method for controlling the remote teaching classroom by the AI and the device for controlling the remote teaching classroom by the AI, when remote teaching is carried out, a lecturer can initiate remote lessons at a first terminal, automatically sign in and automatically monitor the learning state of students by analyzing the behaviors of the students in the lessons by controlling a specific application program pre-installed at the first terminal, the lecturer at the first terminal can interact with the students in the lessons at a second terminal in an audio-video mode in real time, and when a server scores the lessons of the students in the lessons at the second terminal through the behavior characteristics, feedback information is sent to the students and/or teachers according to the scores of the lessons of the students in the lessons. The intelligent remote teaching classroom function control system has the advantages that the teaching achievement of the remote teaching classroom is effectively improved, the function of intelligently controlling the remote teaching classroom can be completed only by installing a specific application program, two cameras and an electronic display screen on the first terminal, the cost of the remote teaching classroom is obviously reduced, and better experience is brought to users.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
fig. 1 schematically illustrates a flow chart of a method for implementing an AI-controlled remote teaching classroom according to an embodiment of the present invention;
fig. 2 schematically shows a first terminal cloud classroom interface schematic according to another embodiment of the invention;
fig. 3-4 schematically illustrate an interface diagram in which a first terminal pulls at least one second terminal into a group according to yet another embodiment of the present invention;
fig. 5 schematically illustrates an apparatus for implementing an AI-controlled remote teaching classroom according to still another embodiment of the present invention;
in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Detailed Description
The principles and spirit of the present invention will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, a method and a device for controlling an AI to control a remote teaching classroom are provided.
In this context, it is to be understood that the terms first terminal and second terminal are referred to as teacher attendance terminal and student attendance terminal. Moreover, any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.
Summary of The Invention
The inventor finds that the existing remote teaching classroom scheme has the following defects: the teacher who speaks who carries out remote teaching can not carry out real-time interaction with the classmates who listen to the class to because student's self-control is relatively poor, and the teacher can not observe the current learning state of student and can lead to the very big reduction of student's of listening to class learning efficiency.
In order to overcome the problems in the prior art, the invention provides a method and a device for controlling a remote teaching classroom by an AI, wherein the method comprises the following steps: connecting the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal; the teacher video picture shot by the first terminal is sent to at least one second terminal for playing, and the student video picture shot by the at least one second terminal is sent to the first terminal for playing; identifying behavioral characteristics in the student video pictures shot by the at least one second terminal; scoring the lecture listening behavior of the corresponding student based on the identified behavior characteristics; and sending feedback information to students whose lecture behavior scores exceed a preset threshold value.
Having described the general principles of the invention, various non-limiting embodiments of the invention are described in detail below.
Application scene overview
The embodiment of the invention can be applied to the scenes of remote teaching classes. For example, a lecture teacher initiates a lecture invitation to at least one second terminal which is previously associated with a first terminal through specific application software installed on the first terminal, the second terminal is a student attending terminal, if the second terminal is preset to automatically accept the lecture invitation initiated by the second terminal, after the first terminal initiates the lecture invitation, the first terminal automatically establishes connection with the at least one second terminal, at the moment, a video picture of the student shot by the at least one second terminal is displayed on a display screen of the first terminal, a video picture of the lecture teacher shot by the first terminal is displayed on a display screen of the second terminal, at the moment, the lecture teacher can interact with the student of the second terminal in real time by controlling the first terminal, and the server can recognize the lecture attending state of the student at the second terminal side through behavior and score the lecture attending state of the student according to the obtained lecture attending state, and a reminder is sent to the classmates with the scores exceeding a preset threshold value, and the class listening state of the student is fed back to the lecturer, so that the lecturer can focus on the student to improve the teaching efficiency. However, those skilled in the art will fully appreciate that the applicable scenarios for embodiments of the present invention are not limited in any way by this framework.
Exemplary method
The method for use according to an exemplary embodiment of the invention is described below with reference to fig. 1-4 in connection with an application scenario. It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present invention, and the embodiments of the present invention are not limited in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.
Referring to fig. 1, a flow diagram of a method for implementing an AI-controlled remote teaching classroom according to one embodiment of the present invention is schematically illustrated. The method comprises the following steps:
s100, connecting the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal.
As an example, the first terminal is a teacher class terminal, the second terminal is a student class terminal, and the first terminal is connected to at least one second terminal, so as to implement the audio/video interaction between the first terminal and the at least one second terminal in a specific manner, that is, the server receives a request, initiated by the teacher through the first terminal, for joining the at least one second terminal, which needs to perform the audio/video interaction, to a group, and then initiates a call to all the second terminals joined to the group. More specifically, as shown in fig. 2, first, before connecting a first terminal with at least one second terminal, a cloud class is established in advance on the first terminal side, a cloud class number is assigned to the cloud class, at least one cloud class is also established in advance for at least one second terminal that needs to establish a connection with the first terminal, and a technician adds all second terminals that may need to establish a connection with the first terminal to a member list of the cloud class established by the first terminal in advance. In a specific example, as shown in fig. 3-4, a technician may create a cloud classroom in advance at a first terminal, and a teacher may first click a "go class" button installed on a specific application program of the first terminal and then click an "add" button to display at least one second terminal, such as a plurality of second terminals named "first grade of schoolchild of yokoku, first grade of benefit primary school of experiment" and "first grade of primary school of southern yang village", pre-stored in a member list of the cloud classroom, and the teacher at the first terminal may manually add the second terminal to a group to which interaction is to be created in the remote classroom of this time, and automatically make an audio/video call request to at least one second terminal that has been added to the group, if the second terminal is pre-set to automatically accept the remote classroom call request, if the first terminal is not set to automatically accept the remote classroom call request in advance, the second terminal side needs to manually confirm whether to accept the classroom call request to complete the audio-video interaction with the first terminal after the first terminal initiates the call request. The method can automatically initiate the remote classroom call request to all the second terminals joining the group, and does not need to sequentially initiate the remote classroom call request to the second conference terminals joining the group, so that the time for initiating the remote classroom call request is greatly reduced, the operation is simple and convenient, and the function of synchronously teaching by one key can be realized.
S110, sending the teacher video picture shot by the first terminal to at least one second terminal for playing, and sending the student video picture shot by the at least one second terminal to the first terminal for playing.
As an example, the specific manner of sending the teacher video image shot by the first terminal to at least one second terminal for playing and sending the student video image shot by the at least one second terminal to the first terminal for playing is that the server is connected to the first terminal and the at least one second terminal respectively, the teacher video image shot by the first terminal is sent to the server and forwarded to the at least one second terminal through the server for playing, and the teacher video image shot by the at least one second terminal is sent to the server and forwarded to the first terminal through the server for playing. Specifically, the first terminal and the second terminal are both connected with the same server in advance, wherein the server may be a cloud formed by a plurality of distributed servers, an audio/video picture shot at the first terminal is automatically transmitted to the server in real time, the server forwards the received audio/video picture transmitted by the first terminal to at least one second terminal connected to the server in real time for displaying the audio/video picture on the at least one second terminal, and vice versa, the audio/video picture shot at the at least one second terminal is automatically transmitted to the server in real time, and the server forwards the received audio/video picture transmitted by the at least one second terminal to the first terminal connected to the server in real time for displaying the audio/video picture on the first terminal. The method can enable the audio and video pictures between the first terminal and the at least one second terminal to be transmitted in real time on one hand, and enable the first terminal and the at least one second terminal to receive the audio and video pictures of each other on the other hand, thereby realizing the function of real-time interaction between the first terminal and the at least one second terminal.
And S120, identifying the behavior characteristics in the student video picture shot by the at least one second terminal.
As an example, the behavioral characteristics include one or a combination of attention direction of the student, limb movement of the student, torso movement of the student, and duration of the movement. The specific way of identifying the behavior characteristics in the student video pictures shot by the at least one second terminal is to segment the video of each student in the student video pictures shot by the at least one second terminal, perform behavior model analysis on the segmented video of each student, and identify the behavior characteristics, wherein the behavior model analysis can be realized in a machine learning manner, for example, by shooting a large number of video clips of the student state of listening to a class, manually scoring or labeling the behavior characteristics presented by the video clips to classify the video clips, and inputting the labeling results and the video clips into the behavior analysis model to train and obtain the behavior analysis model capable of identifying the student state of listening to the class. And automatically identifying or scoring the newly input video clip by using the trained behavior analysis model.
Specifically, because the number of students attending lessons is large at the second terminal side, the video image of the student shot at the second terminal needs to be split first to split the video image containing a plurality of students into a plurality of video images only containing one of the students, and the behavior of the student in the split video images only containing one of the students is analyzed, for example, when the trunk of the student in the obtained split video image moves as a curve and the attention direction of the student is not obtained, the trunk movement and the attention direction of the student are obtained, and the state that the student is lying prone at the moment is judged according to the behavior analysis model identification result; when the limb action of a certain student in the acquired split video picture is dance and the acquired attention direction of the student is east-Zhang-xi, the student is judged to be in a busy state at the moment according to the acquired limb action and attention direction of the student and the recognition result of the behavior analysis model.
In another embodiment, the behavior characteristics of the at least one obtained student attending to class at the second terminal side may be classified, and if the student attending to class is prone according to the behavior analysis model, the behavior characteristics of the student at the moment may be classified as not attending to class, and if the student is alarming according to the behavior analysis model, the student at the moment may be classified as disturbing others.
In another embodiment, after splitting videos of students in the video pictures of the students shot by the second terminal, a face picture of each student is identified according to the video pictures of each split student, and a reminder is sent to students who do not join in a classroom by comparing the face picture of each student with a student list. Specifically, before class, the server performs face recognition on students in video pictures of the students shot by the split second terminals through a face recognition technology, compares the face recognition result with the class lists of the students corresponding to different second terminals stored in the server in advance, determines the list of the members in absence of classes according to the comparison result, and sends class reminding to the personal terminals of the students, and can also display the names of the members in absence of classes on the first terminal and the second terminal, so that a lecturer at the first terminal and the students in absence of classes who are clear from the classmates in class can know the students in absence of classes, and send class reminding to the students in absence of classes. The mode can complete the automatic sign-in function through the face recognition technology.
In another embodiment, after the face picture of each student is identified according to the video pictures of each split student, an instruction that a teacher selects at least one student through a first terminal is received, the video pictures of the selected student are amplified and displayed on the first terminal, other second terminals except the second terminal where the selected student is located are automatically muted, and the interactive pictures of the teacher and the selected student are displayed on at least one second terminal in a split screen mode. Specifically, in the ongoing process of the remote classroom, if the lecturer who is the first terminal wants to ask questions of the students who are listening to the second terminal, the master teacher at the first terminal side may call the name of the student and click on the name of the student in the list of students in the ongoing remote classroom displayed on a specific application installed on the first terminal, then, based on the face recognition technology, the video pictures of the students are displayed on the display screen at the first terminal side by the method, and simultaneously, the enlarged video pictures of the students and the video pictures of the teacher who gives the teacher at the first terminal are correspondingly displayed on the display screens of all the second terminals in a split screen mode of 1 x 1, and at the moment, in order to avoid the interference of the sound of other second terminals on the interaction between the student and the teacher, other second terminals can be automatically muted. The mode can enable the interaction between the teacher and the students to be more visualized, and the teacher can enable the teacher to know the class listening state of the students more clearly through the interaction between the teacher and the students listening to the classes in the classroom.
And S130, scoring the class listening behaviors of the corresponding students based on the identified behavior characteristics.
As an example, the specific way of scoring the lecture listening behavior of the corresponding student based on the identified behavior feature is to score the lecture listening behavior of the student based on the classification and duration of the behavior feature. Specifically, the class-listening behavior of the student is scored according to the type and the duration of the behavior feature of the class-listening student of the second terminal acquired in step S120, for example, if the preset score corresponding to the disturbance of another person is 10 points, the score corresponding to the class-listening non-investment is 5 points, the score of the duration of 0 to 10S is 1 point, and the score of the duration of 10 to 20S is 3 points, and the sum is accumulated, the score of the student is 11 points if the type of the behavior feature of the student of the second terminal acquired is the disturbance of another person and the duration of 9S, and the score of the student is 8 points if the type of the behavior of the student of the second terminal acquired is the class-listening non-investment and the duration of 18S. It should be noted here that the specific score setting is not limited here, and the technician can set the score according to actual needs.
And S140, sending feedback information to students with the class-attending behavior scores exceeding a preset threshold value.
As an example, the specific way of sending the feedback information to the students whose lecture attendance scores exceed the predetermined threshold is to send a reminder to the students whose lecture attendance scores exceed the predetermined threshold through a personal terminal, where the personal terminal includes: the reminding mode can be that a character reminding mode or a voice reminding mode is sent to the personal mobile terminal or the answering machine. In another mode, when the value of the lecture listening behavior of a certain student exceeds a preset threshold value, the feedback information is sent to the student, and meanwhile the feedback information is sent to the teacher who speaks the first terminal. In a specific example, if the preset threshold is 30 minutes, when the score of the lecture listening behavior of a certain student exceeds 30 minutes, the server automatically sends a reminder to the student through the personal terminal of the student, and simultaneously marks the video picture of the student on the screen of the first terminal, wherein the specific marking mode may be that the video picture of the student is circled in red and/or the video picture of the student is displayed on the display screen of the first terminal in an enlarged manner. It should be noted that, the specific marking manner is not limited herein, and the skilled person may set the specific marking manner according to actual requirements. The mode enables the lecturer to clearly know the learning states of a plurality of students at the second terminal by automatically monitoring the learning states of students attending classes and highlighting the students with poor learning states to mark, so that the learning efficiency of the students is improved.
Exemplary devices
Having described the method of the exemplary embodiment of the present invention, next, a schematic diagram of an apparatus for implementing an AI-controlled remote teaching classroom according to an exemplary embodiment of the present invention will be described with reference to fig. 5. The device comprises the following modules:
the connection module 500 connects the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal.
As an example, the first terminal is a teacher class terminal, the second terminal is a student class terminal, and the first terminal is connected to at least one second terminal, so as to implement the audio/video interaction between the first terminal and the at least one second terminal in a specific manner, that is, the server receives a request, initiated by the teacher through the first terminal, for joining the at least one second terminal, which needs to perform the audio/video interaction, to a group, and then initiates a call to all the second terminals joined to the group. More specifically, as shown in fig. 2, first, before connecting a first terminal with at least one second terminal, a cloud class is established in advance on the first terminal side, a cloud class number is assigned to the cloud class, at least one cloud class is also established in advance for at least one second terminal that needs to establish a connection with the first terminal, and a technician adds all second terminals that may need to establish a connection with the first terminal to a member list of the cloud class established by the first terminal in advance. In a specific example, as shown in fig. 3-4, a technician may create a cloud classroom in advance at a first terminal, and a teacher may first click a "go class" button installed on a specific application program of the first terminal and then click an "add" button to display at least one second terminal, such as a plurality of second terminals named "first grade of schoolchild of yokoku, first grade of benefit primary school of experiment" and "first grade of primary school of southern yang village", pre-stored in a member list of the cloud classroom, and the teacher at the first terminal may manually add the second terminal to a group to which interaction is to be created in the remote classroom of this time, and automatically make an audio/video call request to at least one second terminal that has been added to the group, if the second terminal is pre-set to automatically accept the remote classroom call request, if the first terminal is not set to automatically accept the remote classroom call request in advance, the second terminal side needs to manually confirm whether to accept the classroom call request to complete the audio-video interaction with the first terminal after the first terminal initiates the call request. The method can automatically initiate the remote classroom call request to all the second terminals joining the group, and does not need to sequentially initiate the remote classroom call request to the second conference terminals joining the group, so that the time for initiating the remote classroom call request is greatly reduced, the operation is simple and convenient, and the function of synchronously teaching by one key can be realized.
The sending module 510 sends the teacher video image shot by the first terminal to at least one second terminal for playing, and sends the student video image shot by the at least one second terminal to the first terminal for playing.
As an example, the specific manner of sending the teacher video image shot by the first terminal to at least one second terminal for playing and sending the student video image shot by the at least one second terminal to the first terminal for playing is that the server is connected to the first terminal and the at least one second terminal respectively, the teacher video image shot by the first terminal is sent to the server and forwarded to the at least one second terminal through the server for playing, and the teacher video image shot by the at least one second terminal is sent to the server and forwarded to the first terminal through the server for playing. Specifically, the first terminal and the second terminal are both connected with the same server in advance, wherein the server may be a cloud formed by a plurality of distributed servers, an audio/video picture shot at the first terminal is automatically transmitted to the server in real time, the server forwards the received audio/video picture transmitted by the first terminal to at least one second terminal connected to the server in real time for displaying the audio/video picture on the at least one second terminal, and vice versa, the audio/video picture shot at the at least one second terminal is automatically transmitted to the server in real time, and the server forwards the received audio/video picture transmitted by the at least one second terminal to the first terminal connected to the server in real time for displaying the audio/video picture on the first terminal. The method can enable the audio and video pictures between the first terminal and the at least one second terminal to be transmitted in real time on one hand, and enable the first terminal and the at least one second terminal to receive the audio and video pictures of each other on the other hand, thereby realizing the function of real-time interaction between the first terminal and the at least one second terminal.
And the identification module 520 identifies the behavior characteristics in the student video pictures shot by the at least one second terminal.
As an example, the behavioral characteristics include one or a combination of attention direction of the student, limb movement of the student, torso movement of the student, and duration of the movement. The specific way of identifying the behavior characteristics in the student video pictures shot by the at least one second terminal is to segment the video of each student in the student video pictures shot by the at least one second terminal, perform behavior model analysis on the segmented video of each student, and identify the behavior characteristics, wherein the behavior model analysis can be realized in a machine learning manner, for example, by shooting a large number of video clips of the student state of listening to a class, manually scoring or labeling the behavior characteristics presented by the video clips to classify the video clips, and inputting the labeling results and the video clips into the behavior analysis model to train and obtain the behavior analysis model capable of identifying the student state of listening to the class. And automatically identifying or scoring the newly input video clip by using the trained behavior analysis model.
Specifically, because the number of students attending lessons is large at the second terminal side, the video image of the student shot at the second terminal needs to be split first to split the video image containing a plurality of students into a plurality of video images only containing one of the students, and the behavior of the student in the split video images only containing one of the students is analyzed, for example, when the trunk of the student in the obtained split video image moves as a curve and the attention direction of the student is not obtained, the trunk movement and the attention direction of the student are obtained, and the state that the student is lying prone at the moment is judged according to the behavior analysis model identification result; when the limb action of a certain student in the acquired split video picture is dance and the acquired attention direction of the student is east-Zhang-xi, the student is judged to be in a busy state at the moment according to the acquired limb action and attention direction of the student and the recognition result of the behavior analysis model.
In another embodiment, the behavior characteristics of the at least one obtained student attending to class at the second terminal side may be classified, and if the student attending to class is prone according to the behavior analysis model, the behavior characteristics of the student at the moment may be classified as not attending to class, and if the student is alarming according to the behavior analysis model, the student at the moment may be classified as disturbing others.
In another embodiment, after splitting videos of students in the video pictures of the students shot by the second terminal, a face picture of each student is identified according to the video pictures of each split student, and a reminder is sent to students who do not join in a classroom by comparing the face picture of each student with a student list. Specifically, before class, the server performs face recognition on students in video pictures of the students shot by the split second terminals through a face recognition technology, compares the face recognition result with the class lists of the students corresponding to different second terminals stored in the server in advance, determines the list of the members in absence of classes according to the comparison result, and sends class reminding to the personal terminals of the students, and can also display the names of the members in absence of classes on the first terminal and the second terminal, so that a lecturer at the first terminal and the students in absence of classes who are clear from the classmates in class can know the students in absence of classes, and send class reminding to the students in absence of classes. The mode can complete the automatic sign-in function through the face recognition technology.
In another embodiment, after the face picture of each student is identified according to the video pictures of each split student, an instruction that a teacher selects at least one student through a first terminal is received, the video pictures of the selected student are amplified and displayed on the first terminal, other second terminals except the second terminal where the selected student is located are automatically muted, and the interactive pictures of the teacher and the selected student are displayed on at least one second terminal in a split screen mode. Specifically, in the ongoing process of the remote classroom, if the lecturer who is the first terminal wants to ask questions of the students who are listening to the second terminal, the master teacher at the first terminal side may call the name of the student and click on the name of the student in the list of students in the ongoing remote classroom displayed on a specific application installed on the first terminal, then, based on the face recognition technology, the video pictures of the students are displayed on the display screen at the first terminal side by the method, and simultaneously, the enlarged video pictures of the students and the video pictures of the teacher who gives the teacher at the first terminal are correspondingly displayed on the display screens of all the second terminals in a split screen mode of 1 x 1, and at the moment, in order to avoid the interference of the sound of other second terminals on the interaction between the student and the teacher, other second terminals can be automatically muted. The mode can enable the interaction between the teacher and the students to be more visualized, and the teacher can enable the teacher to know the class listening state of the students more clearly through the interaction between the teacher and the students listening to the classes in the classroom.
And the scoring module 530 is used for scoring the class attending behaviors of the corresponding students based on the identified behavior characteristics.
As an example, the specific way of scoring the lecture listening behavior of the corresponding student based on the identified behavior feature is to score the lecture listening behavior of the student based on the classification and duration of the behavior feature. Specifically, the class-listening behavior of the student is scored according to the type and the duration of the behavior feature of the class-listening student of the second terminal acquired in step S120, for example, if the preset score corresponding to the disturbance of another person is 10 points, the score corresponding to the class-listening non-investment is 5 points, the score of the duration of 0 to 10S is 1 point, and the score of the duration of 10 to 20S is 3 points, and the sum is accumulated, the score of the student is 11 points if the type of the behavior feature of the student of the second terminal acquired is the disturbance of another person and the duration of 9S, and the score of the student is 8 points if the type of the behavior of the student of the second terminal acquired is the class-listening non-investment and the duration of 18S. It should be noted here that the specific score setting is not limited here, and the technician can set the score according to actual needs.
The feedback module 540 sends feedback information to students whose lecture behavior score exceeds a predetermined threshold.
As an example, the specific way of sending the feedback information to the students whose lecture attendance scores exceed the predetermined threshold is to send a reminder to the students whose lecture attendance scores exceed the predetermined threshold through a personal terminal, where the personal terminal includes: the reminding mode can be that a character reminding mode or a voice reminding mode is sent to the personal mobile terminal or the answering machine. In another mode, when the value of the lecture listening behavior of a certain student exceeds a preset threshold value, the feedback information is sent to the student, and meanwhile the feedback information is sent to the teacher who speaks the first terminal. In a specific example, if the preset threshold is 30 minutes, when the score of the lecture listening behavior of a certain student exceeds 30 minutes, the server automatically sends a reminder to the student through the personal terminal of the student, and simultaneously marks the video picture of the student on the screen of the first terminal, wherein the specific marking mode may be that the video picture of the student is circled in red and/or the video picture of the student is displayed on the display screen of the first terminal in an enlarged manner. It should be noted that, the specific marking manner is not limited herein, and the skilled person may set the specific marking manner according to actual requirements. The mode enables the lecturer to clearly know the learning states of a plurality of students at the second terminal by automatically monitoring the learning states of students attending classes and highlighting the students with poor learning states to mark, so that the learning efficiency of the students is improved.
It should be noted that although in the above detailed description reference is made to several units/modules or sub-units/modules of the AI control remote teaching classroom, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A method of AI controlling a remote teaching classroom, comprising:
connecting the first terminal to at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal;
the teacher video picture shot by the first terminal is sent to at least one second terminal for playing, and the student video picture shot by the at least one second terminal is sent to the first terminal for playing;
identifying behavioral characteristics in the student video pictures shot by the at least one second terminal;
scoring the lecture listening behavior of the corresponding student based on the identified behavior characteristics;
and sending feedback information to students whose lecture behavior scores exceed a preset threshold value.
2. The method of claim 1, wherein the first terminal is a teacher attendance terminal;
the second terminal is a student class attending terminal.
3. The method according to claim 1, wherein the step of connecting the first terminal to the at least one second terminal to realize audio-video interaction between the first terminal and the at least one second terminal comprises:
receiving a request which is initiated by a teacher through a first terminal and used for adding at least one second terminal needing audio-video interaction into a group;
and initiating a call to all second terminals joined in the group.
4. The method of claim 1, wherein the step of sending the teacher video picture taken by the first terminal to at least one second terminal for playing and the student video picture taken by the at least one second terminal to the first terminal for playing comprises:
respectively connected with a first terminal and at least one second terminal;
sending the video picture of the teacher shot by the first terminal to a server and forwarding the video picture to the at least one second terminal for playing through the server;
and sending the video picture of the teacher shot by the at least one second terminal to a server and forwarding the video picture to the first terminal for playing through the server.
5. The method of claim 1, wherein the step of identifying behavioral features in the student video pictures taken by the at least one second terminal comprises:
segmenting the video of each student in the student video pictures shot by the at least one second terminal;
and performing behavior model analysis on the segmented video of each student, and identifying the characteristic of trip.
6. The method of claim 5, further comprising, after the step of slicing the video of each student in the student video pictures taken by the at least one second terminal:
recognizing the face picture of each student according to the video pictures of each student after segmentation;
comparing the face picture of each student with a student list;
a reminder is sent to students who are not participating in the classroom.
7. The method of claim 6, further comprising, after the step of identifying a face picture of each student from the sliced video pictures of each student:
receiving an instruction of a teacher selecting at least one student through a first terminal;
amplifying the video pictures of the selected students and displaying the video pictures on the first terminal;
automatically muting other second terminals except the second terminal where the selected student is located;
and displaying the interactive pictures of the teacher and the selected students on at least one second terminal in a split screen mode.
8. The method of claim 5, wherein the behavioral characteristics include one or a combination of attention direction of the student, limb movements of the student, torso movements of the student, and duration of the movements.
9. The method of claim 5, wherein after the step of performing behavior model analysis on the segmented video of each student and identifying travel as a feature, the method further comprises:
and classifying the behavior characteristics.
10. An AI-controlled remote teaching classroom apparatus comprising:
the connection module is used for connecting the first terminal to at least one second terminal so as to realize audio and video interaction between the first terminal and the at least one second terminal;
the transmitting module is used for transmitting the teacher video picture shot by the first terminal to at least one second terminal for playing and transmitting the student video picture shot by the at least one second terminal to the first terminal for playing;
the identification module is used for identifying the behavior characteristics in the student video pictures shot by the at least one second terminal;
the scoring module is used for scoring the lecture attending behaviors of the corresponding students based on the identified behavior characteristics;
and the feedback module is used for sending feedback information to students of which the class-attending behavior scores exceed a preset threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011173948.9A CN112185195A (en) | 2020-10-28 | 2020-10-28 | Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011173948.9A CN112185195A (en) | 2020-10-28 | 2020-10-28 | Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112185195A true CN112185195A (en) | 2021-01-05 |
Family
ID=73916083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011173948.9A Pending CN112185195A (en) | 2020-10-28 | 2020-10-28 | Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112185195A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113051020A (en) * | 2021-03-31 | 2021-06-29 | 读书郎教育科技有限公司 | Intelligent classroom system and method capable of realizing live broadcast class reminding |
| CN113490002A (en) * | 2021-05-26 | 2021-10-08 | 深圳点猫科技有限公司 | Interactive method, device, system and medium for online teaching |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9240129B1 (en) * | 2012-05-01 | 2016-01-19 | Google Inc. | Notifications and live updates for braille displays |
| CN106330912A (en) * | 2016-08-25 | 2017-01-11 | 北京智能管家科技有限公司 | Video live broadcasting and two-way communication fusion system and method |
| CN109035089A (en) * | 2018-07-25 | 2018-12-18 | 重庆科技学院 | A kind of Online class atmosphere assessment system and method |
| CN109448477A (en) * | 2018-12-28 | 2019-03-08 | 广东新源信息技术有限公司 | A kind of remote interactive teaching system and method |
| CN109740498A (en) * | 2018-12-28 | 2019-05-10 | 广东新源信息技术有限公司 | A kind of wisdom classroom based on face recognition technology |
| CN110349456A (en) * | 2019-07-12 | 2019-10-18 | 阔地教育科技有限公司 | Intelligence control system, remote control terminal and the classroom terminal of interaction classroom |
-
2020
- 2020-10-28 CN CN202011173948.9A patent/CN112185195A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9240129B1 (en) * | 2012-05-01 | 2016-01-19 | Google Inc. | Notifications and live updates for braille displays |
| CN106330912A (en) * | 2016-08-25 | 2017-01-11 | 北京智能管家科技有限公司 | Video live broadcasting and two-way communication fusion system and method |
| CN109035089A (en) * | 2018-07-25 | 2018-12-18 | 重庆科技学院 | A kind of Online class atmosphere assessment system and method |
| CN109448477A (en) * | 2018-12-28 | 2019-03-08 | 广东新源信息技术有限公司 | A kind of remote interactive teaching system and method |
| CN109740498A (en) * | 2018-12-28 | 2019-05-10 | 广东新源信息技术有限公司 | A kind of wisdom classroom based on face recognition technology |
| CN110349456A (en) * | 2019-07-12 | 2019-10-18 | 阔地教育科技有限公司 | Intelligence control system, remote control terminal and the classroom terminal of interaction classroom |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113051020A (en) * | 2021-03-31 | 2021-06-29 | 读书郎教育科技有限公司 | Intelligent classroom system and method capable of realizing live broadcast class reminding |
| CN113490002A (en) * | 2021-05-26 | 2021-10-08 | 深圳点猫科技有限公司 | Interactive method, device, system and medium for online teaching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104394363B (en) | A kind of Online class director method and system | |
| WO2018072390A1 (en) | Classroom teaching recording and requesting method and system | |
| CN107316520B (en) | Video teaching interaction method, device, equipment and storage medium | |
| CN104735416A (en) | Tracking camera, record information acquisition processing live broadcast network teaching system | |
| CN103413471A (en) | Method and system for controlling network synchronous classroom | |
| CN107731032A (en) | A kind of audio-video switching method, device and remote multi-point interactive education system | |
| CN103413472A (en) | Method and system for achieving network synchronous classroom | |
| CN103414880B (en) | The processing method of a kind of simultaneous classroom and system | |
| CN105654801A (en) | Network teaching method, device and system | |
| CN112652200A (en) | Man-machine interaction system, man-machine interaction method, server, interaction control device and storage medium | |
| CN103414775A (en) | Method and system for achieving network synchronous classroom | |
| CN106454200A (en) | Video interaction method and video interaction system based on scene switching | |
| CN112185195A (en) | Method and device for controlling remote teaching classroom by AI (Artificial Intelligence) | |
| CN111479124A (en) | Real-time playing method and device | |
| CN108847075A (en) | A kind of long-distance educational system and educational method | |
| CN106384546A (en) | Remote interactive teaching automatic voicing method based on students' behaviors | |
| CN110933510B (en) | Information interaction method in control system | |
| CN112837574B (en) | Interactive classroom system and method thereof | |
| CN110971875B (en) | Control method and device combining recording and broadcasting system and IP broadcasting system | |
| US20160372154A1 (en) | Substitution method and device for replacing a part of a video sequence | |
| CN113467604A (en) | Data interaction method and related equipment | |
| CN111081101A (en) | Interactive recording and broadcasting system, method and device | |
| CN110853435A (en) | Remote teaching training system and method | |
| CN115412679B (en) | Interactive teaching quality assessment system with direct recording and broadcasting function and method thereof | |
| CN111031369A (en) | Information screen projection method in control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210105 |