CN112069970B - Classroom teaching event analysis method and device - Google Patents
Classroom teaching event analysis method and device Download PDFInfo
- Publication number
- CN112069970B CN112069970B CN202010899537.1A CN202010899537A CN112069970B CN 112069970 B CN112069970 B CN 112069970B CN 202010899537 A CN202010899537 A CN 202010899537A CN 112069970 B CN112069970 B CN 112069970B
- Authority
- CN
- China
- Prior art keywords
- teaching
- event
- analysis
- events
- classroom
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 claims abstract description 37
- 230000006872 improvement Effects 0.000 claims abstract description 29
- 230000003993 interaction Effects 0.000 claims abstract description 25
- 238000005070 sampling Methods 0.000 claims abstract description 24
- 238000010219 correlation analysis Methods 0.000 claims abstract description 12
- 230000007246 mechanism Effects 0.000 claims abstract description 10
- 230000009471 action Effects 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 24
- 230000006399 behavior Effects 0.000 claims description 22
- 230000003068 static effect Effects 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 11
- 230000002452 interceptive effect Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000013461 design Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 7
- 230000000306 recurrent effect Effects 0.000 claims description 6
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 238000013136 deep learning model Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 238000013518 transcription Methods 0.000 claims description 5
- 230000035897 transcription Effects 0.000 claims description 5
- 238000013473 artificial intelligence Methods 0.000 description 7
- 238000012482 interaction analysis Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012300 Sequence Analysis Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Educational Administration (AREA)
- General Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Economics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Technology (AREA)
- General Business, Economics & Management (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Development Economics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Psychiatry (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
Abstract
The invention discloses a classroom teaching event analysis method and device, wherein the method comprises the following steps: forming a teaching event recognition rule set through correlation analysis and feature matching among multi-source data, recognizing different teaching events through the rule set, and classifying the different teaching events into different teaching stages; generating teaching method structure sequences in different teaching stages; the method comprises the steps of coding speech and behavior interaction in different teaching stages through time sampling; interpreting the analysis results based on evidence according to an educational theory; and generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the interpretation result. The method provides a novel teaching event analysis method based on natural language understanding and computer vision technology, so that time and labor are saved, simplicity and high efficiency are achieved, and the teaching significance in class is focused.
Description
Technical Field
The invention relates to the technical field of data analysis, in particular to a classroom teaching event analysis method and device.
Background
In the traditional quantitative path-taking classroom teaching analysis, the most time-consuming part is manual coding. And the analysis technology mainly based on the time sampling method can lead to the difficulties of splitting teaching situation, complicated analysis, low efficiency, overhigh labor cost input and the like. Therefore, how to utilize new technology to efficiently and standardize the coding of events is a stage in which artificial intelligence can play a role.
Classroom teaching analysis can be divided into quantitative path taking, qualitative path taking and professional growth path taking. The evolution process of the quantitative path-taking analysis technology is most relevant to the technical development, and an S-T teacher-student behavior analysis method and a Frands speech interaction analysis method FIAS based on video tape analysis are generated in sequence. One of the commonalities of the quantitative routing class analysis is time sampling.
The S-T analysis method is to sample and code all the behavior expressions of teachers and students in a class according to the time process. The analysis method takes classroom teaching video as a data source, takes 15 seconds or 30 seconds as a sampling interval, records teachers and students behaviors at the moment of a sampling point, uses an S-T recording card with a horizontal axis of T and a vertical axis of S and the same length as the classroom teaching time, converts data into corresponding line segments according to the recorded sequence of the behavior data of the sampling point, and draws an S-T curve graph. However, both S behavior and T behavior may occur during the sampling, which inevitably hinders the processing of the educational information. And the S-T analysis method only relates to the analysis of the behaviors of teachers and students, and does not consider the verbal interaction in the classroom teaching process.
The frand speech interactive Analysis coding System (fians interactive Analysis System, FIAS for short) samples and analyzes all speech interactions of teachers and students in a class according to the time course. FIAS divides teacher and student speech into teacher speaking, student speaking, and no person speaking/disorder, the teacher speaking is divided into two 7 kinds of speaking codes with direct influence and indirect influence, the student speaking is divided into 2 kinds of passive speaking and active speaking, and 10 kinds of speech codes are added under the condition of silence/disorder. FIAS takes 3 seconds as a sampling interval, encodes the speech of the classroom teaching video one by one according to the time sequence, and connects the speech into a sequence to analyze different classroom teaching structures, modes and styles. However, the information technology element in the classroom teaching is ignored, and it is difficult to perform the sampling and encoding operation every 30 seconds.
The disadvantages of classroom teaching analysis based on time sampling methods are the lack of data structure and sense understanding, the time-consuming and inefficient manual analysis, and the need to rely entirely on experienced researchers, etc.
First, the lack of association of data structures with sense understanding
The existing time sampling method for analysis is to see the whole class as an analysis object, and put the times and duration of each type of speech or behavior interaction into the whole class for statistics on average, so that each meaningful speech interaction or behavior interaction is distributed to all teaching time, the connection between a data structure and meaning understanding is not easy to establish, and a teacher is not easy to help to understand the activity arrangement corresponding to the learner cognitive processing process more deeply, and the improvement is also carried out.
Second, manual analysis is time consuming and inefficient
Taking a lesson of 40 minutes as an example, the S-T method cuts speech in 15 seconds, the FIAS cuts speech in 3 seconds, 160 or 800 codes are generated, and the codes can be completed after repeated confirmation by professionals. Sampling and encoding every 30 seconds is difficult to operate, and is also done manually by human experts.
Thirdly, relying entirely on experienced researchers
Classroom teaching analysis work must be performed by trained personnel and at least two researchers are required to complete each case. The manual analysis cost of the unit case is high, the capacity of the research sample is limited, and the analysis is time-consuming and inefficient. Therefore, the classroom teaching analysis technology mainly based on the time sampling method has high dependence on professionals, is difficult to form large-scale and rapid batch analysis, is difficult to meet a great amount of requirements of classroom analysis, and the like.
Generally, the time sampling method is still in the technical routes of manual coding and tool statistics, and has not made breakthrough progress in the aspects of research thought and efficiency.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, one objective of the present invention is to provide a classroom teaching event analysis method, which provides a novel teaching event analysis method based on natural language understanding and computer vision technology, so as to achieve the purposes of time saving, labor saving, simplicity, high efficiency, and focusing on classroom teaching significance.
The invention also aims to provide a classroom teaching event analysis device.
In order to achieve the above object, an embodiment of the present invention provides a classroom teaching event analysis method, including:
s1, forming a teaching event recognition rule set through correlation analysis and feature matching among multi-source data, recognizing different teaching events through the rule set, and classifying the different teaching events into different teaching stages;
s2, generating teaching method structure sequences in different teaching stages;
s3, carrying out coding on the speech and behavior interaction in different teaching stages through time sampling;
s4, according to an educational theory, interpreting an analysis result based on evidence;
and S5, generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the interpretation result.
In order to achieve the above object, another embodiment of the present invention provides an apparatus for analyzing classroom teaching events, including:
the identification module is used for forming a teaching event identification rule set through correlation analysis and feature matching among multi-source data, identifying different teaching events through the rule set and classifying the teaching events into different teaching stages;
the sequencing module is used for generating the sequencing of the teaching method structures in different teaching stages;
the internal analysis module is used for coding the speech and behavior interaction in different teaching stages through time sampling;
the interpretation module is used for interpreting the analysis result based on the evidence according to the educational theory;
and the improvement module is used for generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the reading result.
According to the classroom teaching event analysis method and device provided by the embodiment of the invention, classroom teaching events are taken as research objects, and an artificial intelligence technology is utilized, so that standardized, calculable, large-scale and efficient classroom teaching analysis is realized, the dilemma that the analysis is complicated and low-efficiency, the classroom situation is split, the time and labor cost input is too high and the like caused by adopting a time sampling method are overcome, and the classroom teaching significance is focused.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow diagram of a classroom teaching event analysis method according to one embodiment of the present invention;
FIG. 2 is a time distribution plot of a sequence and a teaching event according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of computer vision based classroom behavior analysis in accordance with one embodiment of the present invention;
FIG. 4 is a diagram of a classroom teaching event analysis framework according to one embodiment of the present invention;
fig. 5 is a schematic structural diagram of a classroom teaching event analysis device according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
Hereinafter, a classroom teaching event analysis method and apparatus according to an embodiment of the present invention will be described with reference to the accompanying drawings.
First, a method for analyzing a classroom teaching event according to an embodiment of the present invention will be described with reference to the accompanying drawings.
Fig. 1 is a flowchart of a classroom teaching event analysis method according to an embodiment of the present invention.
As shown in fig. 1, the classroom teaching event analysis method includes the following steps:
s1, forming a teaching event recognition rule set through correlation analysis and feature matching among multi-source data, recognizing different teaching events through the rule set, and classifying the different teaching events into different teaching stages.
Further, in one embodiment of the present invention, S1 further comprises: through correlation analysis and feature matching among multi-source data, key short texts in teaching design and key scene and action recognition in videos are established, interactive modes in the texts are recorded to switch keywords, a teaching event recognition rule set is formed, recognition of different teaching events is carried out, and the teaching events are classified into different teaching stages.
Further, in an embodiment of the present invention, S1 further includes:
s11, extracting teaching events in a classroom teaching transcription text, classifying the teaching events, performing characteristic analysis on each teaching event, giving a label of each teaching event, noting the category of the teaching event, labeling the labels of the teaching events, and taking the labeled data as a training set;
and S12, obtaining a weight matrix of the word vector through deep learning model training, inputting the weight matrix and a training set into a recurrent neural network for model training to obtain an event classifier, and identifying the event by using the event classifier through the event classifier.
Specifically, in the traditional quantitative path-taking classroom teaching analysis, the most time-consuming part is manual coding. The novel technology is utilized to carry out efficient and standardized coding on events, the stages in the teaching design scheme are divided to be used as references for teaching event identification, then the key scenes and the change of the actions of teachers and students are searched in the time periods before and after the corresponding nodes in the classroom video, or the text is recorded in the classroom teaching, keywords for switching the interactive modes of the actions or the words are searched, the key short texts in the teaching design, the recognition of the key scenes and the actions in the video, the keywords for switching the interactive modes in the recorded text and the like are established through the correlation analysis and the characteristic matching among multi-source data, a teaching event identification rule set is formed, and the accurate identification of different teaching events is carried out.
Furthermore, a natural language understanding technology is used for carrying out relationship evaluation, identifying and analyzing teaching events and sequences, learning and partially replacing traditional manual codes so as to assist human experts in classroom teaching analysis.
Firstly, label division and text labeling are carried out on teaching events. The teaching events in the classroom teaching transcription text are extracted, the events are classified, and the teaching events need to be understood by people, including dividing labels and text labels. In the stage of dividing the labels, feature analysis of each teaching event is mainly completed, and labels are given to a sentence or a few sentences to indicate which teaching event belongs to. And in the text labeling stage, labeling the division labels, wherein the labeled data is used for training the classifier.
Secondly, based on event classifier of GRU recurrent neural network. The learning of words means that the task is to implement the conversion of words appearing in the text into word vectors. The representation of word vectors is an important link in natural language processing. The method adopts a deep learning model Word2vec, obtains a weight matrix of Word vectors through training, inputs the weight matrix into an Embellding layer of a recurrent neural network for model training, outputs a classification result and finally judges whether the model training is successful or not through the accuracy of a training set and a testing set by using an average hidden state in each layer of the network. According to the event classification result, the start and stop time points of the divided events are combined to generate the teaching event type and the time distribution map of each class, as shown in fig. 2.
And S2, generating teaching method structure sequences in different teaching stages.
In technical implementation, a teaching method structure sequence classifier is established by adopting a natural language understanding technology. The input to the classifier is text type data. Since the text data contains context information, attention is drawn to better understand the meaning of a sentence or word in the input data. In the text classification, a sentence core word and an event core sentence are determined using an attention mechanism. And respectively modeling sentences and chapters in the text data to provide a teaching method structure sequence.
And S3, interactively coding the speech and the behavior in different teaching stages through time sampling.
Further, in an embodiment of the present invention, S3 further includes:
counting and classifying the key words in each teaching stage by a natural language understanding technology, and performing speech interaction analysis in the teaching stages by adopting the standard one-to-one correspondence in a coding system;
the skeleton change of teachers and students is judged through a computer vision technology, and analysis is carried out according to the skeleton change.
Further, in an embodiment of the present invention, S3 further includes:
s31, acquiring a static video frame, and carrying out scene classification on the static video frame;
s32, after the scenes are classified, identifying key interaction equipment in the teaching video by using a target detection method;
and S33, analyzing the skeleton key sequence characteristics based on an OpenPose framework by utilizing airspace characteristic analysis, performing a real-time skeleton key human body posture prediction algorithm, surrounding a classroom scene on the basis of a static image action recognition algorithm, and recognizing and counting the student action in a teaching video.
Specifically, when all the classification, identification and sequence of the teaching events in a class are analyzed, the inter-event analysis can be performed. For speech interaction analysis in a classroom, a natural language understanding technology can be adopted to count and classify key words in each teaching event, and the speech interaction analysis in the event is completed by adopting the standard one-to-one correspondence in coding systems such as ITIAS and the like; for behavior interaction analysis in a video, computer vision can judge obvious skeleton change generated when teachers and students stand, raise hands, walk, write on a blackboard or operate a tablet computer in a teaching video, statistics of data such as teacher-student interaction, man-machine interaction, student interaction and the like in a picture is carried out according to the obvious skeleton change, then analysis data is calculated in real time by using a video analysis report template with specific analysis dimensionality, and an artificial intelligence analysis result with high accuracy and strong robustness (Robust) is formed by classroom teaching analysis which can be carried out in real time and on a large scale.
Further, computer-vision classroom teaching behavior analysis is employed in this step. The teaching scenes in the classroom video are classified firstly, then key interaction equipment such as an electronic whiteboard and a handheld flat plate are distinguished, and the classroom teacher and student behaviors are automatically identified and calculated in batches.
Specifically, firstly, scenes are preliminarily classified according to video static frames, 10000 static images are randomly selected from actual classroom teaching videos, manual classification is carried out according to three types of scenes defined in advance, including a teacher teaching scene, a teacher-student interaction scene and a student learning scene, and then a ResNet50 convolution neural network structure is used for completing batch scene classification by a computer.
Secondly, detecting key interaction equipment in the video in an auxiliary mode through a target detection method. After scene identification is completed, key interaction equipment which influences action interaction of teachers and students, such as a blackboard, an electronic whiteboard, a desktop computer, a handheld learning terminal and the like, is identified.
And finally, identifying action behaviors based on the deep convolutional neural network. In the classroom teaching video, the main subjects of behavioral analysis are people, teachers who perform teaching, and students. The motion of a person is a set of motions based on a series of skeletal key points. If a traditional optical flow method is adopted, a lot of non-human motion information can be generated, so that an OpenPose framework-based skeleton key sequence analysis feature and real-time skeleton key human body posture prediction algorithm are adopted in space domain feature analysis. On the basis of a static image action recognition algorithm, the actions of students are recognized and counted in a video around a classroom scene.
The classroom behavior analysis system is established, and realizes modules of key frame extraction, student tracking, action recognition, action statistics and the like, as shown in fig. 3. Firstly, the key frame extraction module adopts a blocking color histogram method to measure the similarity between frames, thereby reducing the redundant information of the video and improving the operation speed. And then, the action recognition module sends the key frame to a double-flow network action recognition framework for analysis. And then, the student tracking module binds the action recognition result with the skeleton key point sequence by adopting a lightweight student tracking method based on the skeleton key points. Finally, the action statistical module is used for teaching and guiding teachers; students stand up, answer questions, and the like. The classroom behavior analysis system can upload a section of classroom teaching video through the client, the server side obtains the video and then operates in the cloud, and the identification result is returned to the client, so that teachers and students' actions are identified and counted, and classroom teaching analysis is assisted.
And S4, according to an educational theory, interpreting the analysis result based on the evidence.
The interpretability is a core appeal of future artificial intelligence development and is a premise of mutual trust of human and machine. For teachers, the design ability, the implementation ability, the thinking resistance and the like of self classroom teaching can be found and compensated in time by means of visual interpretable results; for teaching researchers, intervention and guidance strategies for teaching of teachers can be adjusted in time by means of transverse comparison of analysis results among different teachers and longitudinal comparison of analysis results of the same teacher in different stages; for the education managers, by means of the analysis reports, teacher growth incentive mechanisms, teacher professional development schemes and the like are made or improved more scientifically.
And S5, generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the interpretation result.
The existing teaching event improvement has the following three limitations. Firstly, the classroom teaching improvement strategy has fewer professional groups and a small total number of people, and is not easy to collect a large number of high-quality, targeted and feasible improvement strategies in a short time; secondly, the classroom teaching improvement strategy lacks a relatively standard knowledge expression structure; third, there is a lack of platforms and tools to converge on existing classroom teaching improvement strategies. Therefore, classroom teaching analysis and improvement supported by artificial intelligence, and challenges and opportunities coexist. The embodiment of the invention gradually realizes the improvement of the teaching event by using a three-step method of manual generation, machine generation and man-machine cooperation generation.
The first step is as follows: artificially generated
The existing data set is 638 classroom teaching cases of primary school four-grade mathematics, scores and comments are given to each case by 56 experts in the country, and a plurality of teaching improvement strategies are arranged in the existing data set. Subject experts of 3-5 people organize the team to carry out consistency analysis on classroom teaching event characteristics and teaching improvement strategies, if the strategy is judged to be effective, the strategy is stored in the system in a text format to represent that the strategy can be used subsequently, and if the strategy is judged to be poor in effect, the strategy is selected by an expert team from an existing strategy library or is automatically compiled into a more appropriate improvement strategy, or is stored in the system in a text format to realize the improvement strategy.
The second step is that: machine generation
Because different classroom interaction characteristics and teaching improvement strategies given by human experts are stored in the system, classroom teaching case data with larger sample size are used for training a machine to perform automatic generation. As the existing strategy library is in a text format, in a machine recommendation stage, a knowledge base filling technology ProjE suitable for short texts and an improved algorithm thereof are mainly adopted to realize structured extraction of a classroom teaching improved strategy, then, the strategy is stored in an RDF format and the like, the structured strategy is taken as a label, classroom interaction characteristic analysis results are taken as input, a supervised learning model is established, a teaching improved strategy intelligent classifier is trained through historical data, and a new analysis result automatically predicts an effective classroom teaching improved strategy.
The third step: man-machine collaborative generation
Subsequently, when the system collects enough classroom teaching cases and the accumulated data reaches a larger magnitude, it is highly likely that teaching improvement strategies which are not mentioned before are generated. The method comprises the steps of performing true discovery (Truth discovery) on a crowdsourcing data set of a teaching improvement strategy accumulated by a system by adopting a crowdsourcing technology, maximizing the consistency of the effectiveness of the improvement strategy, preliminarily extracting an effective improvement strategy which is automatically identified by a machine and is generated by the machine and is suitable for different classroom interaction characteristics, matching text pairs by using massive teaching event characteristics and teaching improvement strategies accumulated by the system, automatically generating a teaching improvement strategy scheme in a text form by adopting an end-to-end learning algorithm based on a deep neural network, learning and revising principles by the machine again after requesting human experts to perform sampling verification and revision, and realizing maximum artificial participation by multiple iterations to generate human-computer collaborative teaching event improvement.
According to the classroom teaching event analysis method provided by the embodiment of the invention, classroom teaching events are taken as research objects, and an artificial intelligence technology is utilized to realize standardized, calculable, large-scale and efficient classroom teaching analysis, so that the difficulties of complicated and low-efficiency analysis, classroom situation cutting, high investment of time and labor cost and the like caused by adopting a time sampling method are overcome, and the classroom teaching significance is focused.
Next, a classroom teaching event analysis apparatus proposed according to an embodiment of the present invention will be described with reference to the drawings.
Fig. 5 is a schematic structural diagram of a classroom teaching event analysis device according to an embodiment of the invention.
As shown in fig. 5, the classroom teaching event analysis device includes: an identification module 701, a ranking module 702, an internal analysis module 703, an interpretation module 704, and an improvement module 705.
The identification module 701 is configured to form a teaching event identification rule set through correlation analysis and feature matching between multi-source data, identify different teaching events through the rule set, and classify the different teaching events into different teaching stages.
And the sequencing module 702 is used for generating the teaching method structure sequencing in different teaching stages.
An internal analysis module 703 for encoding speech and behavior interactions in different teaching stages by time sampling.
An interpretation module 704 for interpreting the analysis results based on the evidence according to an educational theory.
And the improvement module 705 is used for generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the interpretation result.
Further, in an embodiment of the present invention, the recognition module is specifically configured to establish recognition of key short texts and key scenes and actions in videos in the teaching design through correlation analysis and feature matching between multi-source data, switch keywords in an interactive manner in the recorded text, form a teaching event recognition rule set, perform recognition of different teaching events, and classify the teaching events into different teaching stages.
Further, in one embodiment of the present invention, the identification module includes:
the labeling unit is used for extracting the teaching events in the classroom teaching transcription text, classifying the teaching events, performing characteristic analysis on each teaching event, giving a label of each teaching event, noting the category of the teaching event, labeling the labels of the teaching events, and taking the labeled data as a training set;
and the training unit is used for obtaining a weight matrix of the word vector through deep learning model training, inputting the weight matrix and the training set into a recurrent neural network for model training to obtain an event classifier, and identifying the event by using the event classifier through the event classifier.
Further, in an embodiment of the present invention, the internal analysis module is specifically configured to count and classify the key words in each teaching stage through a natural language understanding technology, and perform speech interaction analysis in the teaching stage by using a one-to-one correspondence between standards in a coding system; the skeleton change of teachers and students is judged through a computer vision technology, and analysis is carried out according to the skeleton change.
Further, in one embodiment of the present invention, the internal analysis module comprises:
the acquisition unit is used for acquiring the static video frame and carrying out scene classification on the static video frame;
the detection unit is used for identifying key interaction equipment in the teaching video by using a target detection method after scene classification;
and the statistical unit is used for analyzing the bone key sequence characteristics based on an OpenPose framework by utilizing spatial domain characteristic analysis, performing a real-time bone key human body posture prediction algorithm, surrounding a classroom scene on the basis of a static image action recognition algorithm, and recognizing and counting the actions of students in a teaching video.
It should be noted that the foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and is not repeated herein.
According to the classroom teaching event analysis device provided by the embodiment of the invention, classroom teaching events are taken as research objects, and an artificial intelligence technology is utilized to realize standardized, calculable, large-scale and high-efficiency classroom teaching analysis, so that the difficulties of complicated and low-efficiency analysis, classroom situation cutting, high time and labor cost input and the like caused by adopting a time sampling method are overcome, and the classroom teaching event analysis device is more focused on classroom teaching significance.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless explicitly specified otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (3)
1. A classroom teaching event analysis method is characterized by comprising the following steps:
s1, forming a teaching event recognition rule set through correlation analysis and feature matching among multi-source data, recognizing different teaching events through the rule set, and classifying the different teaching events into different teaching stages;
s2, generating teaching method structure sequences in different teaching stages;
s3, coding the speech and behavior interaction in different teaching stages through time sampling;
s4, according to an educational theory, interpreting an analysis result based on evidence;
s5, generating an optimized classroom teaching improvement mechanism through man-machine cooperation according to the interpretation result;
the S1 further includes: establishing key short texts in teaching design and key scenes and action recognition in videos through correlation analysis and feature matching among multi-source data, recording interactive mode switching keywords in the texts to form a teaching event recognition rule set, recognizing different teaching events, and classifying the teaching events into different teaching stages;
the S1 further includes:
s11, extracting teaching events in a classroom teaching transcription text, classifying the teaching events, performing characteristic analysis on each teaching event, giving a label of each teaching event, noting the category of the teaching event, labeling the labels of the teaching events, and taking the labeled data as a training set;
s12, obtaining a weight matrix of a word vector through deep learning model training, inputting the weight matrix and a training set into a recurrent neural network for model training to obtain an event classifier, and performing event identification by using the event classifier through the event classifier;
the S2 further comprises: establishing a teaching method structure sequence classifier according to a natural language understanding technology, wherein the input of the teaching method structure classifier is text type data, sentence core words and event core sentences in the text type data are determined through an attention mechanism, and a teaching method structure sequence is generated by respectively modeling sentences and chapters in the text type data;
the S3 further includes:
counting and classifying the key words in each teaching stage by a natural language understanding technology, and performing speech interactive analysis in the teaching stages by adopting the standard one-to-one correspondence in a coding system;
and judging the skeleton change of teachers and students by a computer vision technology, and analyzing according to the skeleton change.
2. The classroom teaching event analysis method as claimed in claim 1, wherein said S3 further comprises:
s31, acquiring a static video frame, and carrying out scene classification on the static video frame;
s32, after the scenes are classified, identifying key interaction equipment in the teaching video by using a target detection method;
and S33, analyzing the skeleton key sequence characteristics based on an OpenPose framework by utilizing space domain characteristic analysis, performing a real-time skeleton key human body posture prediction algorithm, surrounding a classroom scene on the basis of a static image action recognition algorithm, and recognizing and counting the actions of students in a teaching video.
3. The utility model provides a classroom teaching incident analytical equipment which characterized in that includes:
the identification module is used for forming a teaching event identification rule set through correlation analysis and feature matching among multi-source data, identifying different teaching events through the rule set and classifying the teaching events into different teaching stages;
the sequencing module is used for generating the sequencing of the teaching method structures in different teaching stages;
the internal analysis module is used for coding the speech and behavior interaction in different teaching stages through time sampling;
the interpretation module is used for interpreting the analysis result based on the evidence according to the educational theory;
the improved module is used for generating an optimized classroom teaching improved mechanism through man-machine cooperation according to the reading result;
the recognition module is specifically used for establishing key short texts in teaching design and key scene and action recognition in videos through correlation analysis and feature matching among multi-source data, switching keywords in interactive modes in recorded texts to form a teaching event recognition rule set, recognizing different teaching events and classifying the different teaching events into different teaching stages;
the identification module comprises:
the labeling unit is used for extracting the teaching events in the classroom teaching transcription text, classifying the teaching events, performing characteristic analysis on each teaching event, giving a label of each teaching event, noting the category of the teaching event, labeling the labels of the teaching events, and taking the labeled data as a training set;
the training unit is used for obtaining a weight matrix of a word vector through deep learning model training, inputting the weight matrix and a training set into a recurrent neural network for model training to obtain an event classifier, and utilizing the event classifier to carry out event identification through the event classifier;
the internal analysis module is specifically used for counting and classifying the key words in each teaching stage through a natural language understanding technology, and performing speech interactive analysis in the teaching stages by adopting the standard one-to-one correspondence in a coding system; and judging the skeleton change of teachers and students by a computer vision technology, and analyzing according to the skeleton change.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010899537.1A CN112069970B (en) | 2020-08-31 | 2020-08-31 | Classroom teaching event analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010899537.1A CN112069970B (en) | 2020-08-31 | 2020-08-31 | Classroom teaching event analysis method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112069970A CN112069970A (en) | 2020-12-11 |
| CN112069970B true CN112069970B (en) | 2022-10-04 |
Family
ID=73665184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010899537.1A Active CN112069970B (en) | 2020-08-31 | 2020-08-31 | Classroom teaching event analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069970B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052461B (en) * | 2021-03-25 | 2022-04-08 | 重庆科技学院 | Intelligent text teaching evaluation method based on probability graph mode inference |
| CN113657152A (en) * | 2021-07-07 | 2021-11-16 | 国网江苏省电力有限公司电力科学研究院 | Classroom student behavior recognition system construction method |
| CN113743250B (en) * | 2021-08-16 | 2024-02-13 | 华中师范大学 | Construction method and system of classroom teaching behavior event description model |
| CN115205725B (en) * | 2022-02-22 | 2023-10-27 | 广州云智达创科技有限公司 | Video scene analysis method, device and storage medium |
| CN115052194B (en) * | 2022-06-02 | 2023-05-02 | 北京新唐思创教育科技有限公司 | Learning report generation method, device, electronic equipment and storage medium |
| CN115909152B (en) * | 2022-11-16 | 2023-08-29 | 北京师范大学 | Intelligent teaching scene analysis system based on group behaviors |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740446A (en) * | 2018-12-14 | 2019-05-10 | 深圳壹账通智能科技有限公司 | Classroom students ' behavior analysis method and device |
| CN111027584A (en) * | 2019-10-23 | 2020-04-17 | 宋飞 | Classroom behavior identification method and device |
| CN111046823A (en) * | 2019-12-19 | 2020-04-21 | 东南大学 | Student classroom participation degree analysis system based on classroom video |
| CN111401797A (en) * | 2020-05-09 | 2020-07-10 | 华南师范大学 | Teaching quality evaluation method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9779084B2 (en) * | 2013-10-04 | 2017-10-03 | Mattersight Corporation | Online classroom analytics system and methods |
-
2020
- 2020-08-31 CN CN202010899537.1A patent/CN112069970B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740446A (en) * | 2018-12-14 | 2019-05-10 | 深圳壹账通智能科技有限公司 | Classroom students ' behavior analysis method and device |
| CN111027584A (en) * | 2019-10-23 | 2020-04-17 | 宋飞 | Classroom behavior identification method and device |
| CN111046823A (en) * | 2019-12-19 | 2020-04-21 | 东南大学 | Student classroom participation degree analysis system based on classroom video |
| CN111401797A (en) * | 2020-05-09 | 2020-07-10 | 华南师范大学 | Teaching quality evaluation method and system |
Non-Patent Citations (5)
| Title |
|---|
| OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields;Zhe Cao等;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》;20190530;第1-14页 * |
| 基于智慧课堂的教学行为数据分析研究;周顺等;《中国教育信息化》;20200605(第11期);第75-79页 * |
| 基于课堂视频的学生课堂参与度分析;缪佳等;《教育生物学杂志》;20191215(第04期);第26-32页 * |
| 课堂教学大数据采集与分析平台架构及关键技术研究;辛继胜等;《广东轻工职业技术学院学报》;20200330(第01期);第15-21页 * |
| 课堂环境下基于教学行为的教师教学投入度评测建模研究;牟智佳等;《现代远距离教育》;20200629(第03期);第62-70页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112069970A (en) | 2020-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069970B (en) | Classroom teaching event analysis method and device | |
| El Aissaoui et al. | Combining supervised and unsupervised machine learning algorithms to predict the learners’ learning styles | |
| CN111915148B (en) | Classroom teaching evaluation method and system based on information technology | |
| CN109214664B (en) | Emotional behavior comprehensive analysis system based on artificial intelligence | |
| US10269258B2 (en) | System for automatic assessment of student learning | |
| CN111915460A (en) | AI vision-based intelligent scoring system for experimental examination | |
| CN111524578B (en) | Psychological assessment device, method and system based on electronic psychological sand table | |
| CN115239527A (en) | Teaching behavior analysis system for teaching characteristic fusion and modeling based on knowledge base | |
| CN113408852B (en) | Meta-cognition ability evaluation model based on online learning behavior and deep neural network | |
| CN113744872A (en) | Intelligent interactive Alzheimer's disease auxiliary screening system | |
| CN114913729A (en) | Question selection method and device, computer equipment and storage medium | |
| CN115810163B (en) | Teaching evaluation method and system based on AI classroom behavior recognition | |
| CN116050892A (en) | Intelligent education evaluation supervision method based on artificial intelligence | |
| CN110956142A (en) | Intelligent interactive training system | |
| CN114662779A (en) | Student portrait-based programming score prediction method and system | |
| CN116226410B (en) | Teaching evaluation and feedback method and system for knowledge element connection learner state | |
| CN113888757A (en) | Examination paper intelligent analysis method, examination paper intelligent analysis system and storage medium based on benchmarking evaluation | |
| Stefanova et al. | Educational data mining perspectives within university big data environment | |
| CN116434341A (en) | Student classroom abnormal behavior identification method and system | |
| CN114971425A (en) | Database information monitoring method, device, equipment and storage medium | |
| CN114819304A (en) | NLP-based interviewing process double-group association evaluation method | |
| CN114638988A (en) | Teaching video automatic classification method and system based on different presentation modes | |
| CN114330943A (en) | Interview assisting method and device | |
| Zheng et al. | Automated Multi-Mode Teaching Behavior Analysis: A Pipeline Based Event Segmentation and Description | |
| Dodiya et al. | Attention, emotion and attendance tracker with question generation system using deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |