CN109035089A - A kind of Online class atmosphere assessment system and method - Google Patents

A kind of Online class atmosphere assessment system and method Download PDF

Info

Publication number
CN109035089A
CN109035089A CN201810822970.8A CN201810822970A CN109035089A CN 109035089 A CN109035089 A CN 109035089A CN 201810822970 A CN201810822970 A CN 201810822970A CN 109035089 A CN109035089 A CN 109035089A
Authority
CN
China
Prior art keywords
student
classroom
mood
class
atmosphere
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810822970.8A
Other languages
Chinese (zh)
Inventor
葛继科
刘灿
陈栋
陈祖琴
陈国荣
杜晓霞
罗建伟
任虹
刘垚
何宏黎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Science and Technology
Original Assignee
Chongqing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Science and Technology filed Critical Chongqing University of Science and Technology
Priority to CN201810822970.8A priority Critical patent/CN109035089A/en
Publication of CN109035089A publication Critical patent/CN109035089A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G06Q50/205—Education administration or guidance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Health & Medical Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Data Mining & Analysis (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • General Business, Economics & Management (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

It includes video flowing acquisition module, data flow processing module, image analysis module, class attendance analysis module, classroom atmosphere evaluation module, classroom atmosphere grading module, display module that the present invention, which provides a kind of Online class atmosphere assessment system and method, assessment system,；The present invention acquires classroom video stream data by camera, image one by one is intercepted to the video of acquisition, divided ownership facial image, and it numbers in order, characteristic value is assigned to the face accordingly numbered simultaneously, then recognition of face and human facial expression recognition are carried out by number again, to identify the number of video stream data middle school student, mood and movement posture, it is 0 point that image middle school student, which bow and then score, student has mutual-action behavior to add 1 point, mood analysis strategy when listening to the teacher further according to student show that current student listens to the teacher condition grading, final comprehensive assessment goes out classroom atmosphere scoring.The present invention can assess Classroom instruction quality in real time online, can effectively improve Evaluated effect.

Description

A kind of Online class atmosphere assessment system and method
Technical field
The present invention relates to field of Educational Technology, specially a kind of Online class atmosphere assessment system and method.
Background technique
With the appearance of the universal and various recreational facilities of university education, university student classroom generally occurs some being unfavorable for teaching The case where, such as student attend class and play mobile phone, have on earphone and listen to music, stare blankly, sleep, and this status has seriously affected classroom religion Learn quality.Since students ' interest of study is low, relevant information of problems during giving lessons can not be timely feedbacked to the teacher that attends class, Student is awarded along with part teacher can not more say the interested content of student, causes classroom instruction atmosphere poor, class Hall quality of instruction decline is serious.Based on the above issues, in order to promote Classroom Teaching, teacher is improved to classroom instruction process Control ability, while meeting the needs of educational administration personnel assess teachers ' teaching quality, propose that a set of Online class atmosphere is commented Estimate system, this for teacher reinforce class management, improve the attraction of course teaching, the good study habit of training student and The autocontrol force for promoting student all has important realistic meaning.
Summary of the invention
For the defects in the prior art, the present invention provides a kind of Online class atmosphere assessment system and method, effectively mentions The learning interest of high quality of instruction and student.
Technical solution of the present invention is as follows:
A kind of Online class atmosphere assessment system, it is critical that including:
Video flowing acquisition module, the video flowing acquisition module teach indoor video stream data for acquiring；
Data flow processing module, the data flow processing module are schemed one by one for being intercepted to the video of acquisition Picture；
Image analysis module, the image analysis module for handling image information, obtain in image all face locations and Characteristic value is extracted, segmentation face simultaneously numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify video fluxion According to the number, mood and movement posture of middle school student, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest；
Class attendance analysis module, the class attendance analysis module are used for according to the number of student statistic rate of attendance；
Classroom atmosphere evaluation module, the classroom atmosphere evaluation module are used for according to student's mood and student's movement posture, system Bow number, classroom interactions' number and student of meter student listens to the teacher rate；
Classroom atmosphere grading module, the classroom atmosphere grading module are used to carry out Classroom Teaching according to assessment data Scoring, the assessment packet include bow number, classroom interactions' number and student of student and listen to the teacher rate；
Display module, the display module is for showing the concrete condition that class attendance situation and classroom are attended class, the letter of displaying Breath includes teach teacher, class, class's total number of persons, class period section, student attendance rate and classroom atmosphere appraisal result.
A kind of Online class atmosphere appraisal procedure, which comprises the following steps:
A. classroom video stream data acquires, and acquires video flowing in classroom by camera；
B. video stream data is handled, and is intercepted image one by one to the video of acquisition；
C. Mask-RCNN model divided ownership facial image is used, all face locations in image are obtained and extracts feature Value, segmentation face simultaneously numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify video stream data middle school student Number, mood and movement posture, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest, specifically:
D. according to number of student, the statistic rate of attendance；
E. according to student's mood and student's movement posture, bow rate, classroom interactions' number and student of statistic listens to the teacher rate；
F. interval time, according to assessment data score classroom atmosphere, the assessment packet include student bow rate, Classroom interactions' number and student listen to the teacher rate:
F1. image information is analyzed, student bows, which is divided into 0 point, and student does not occur bowing executing step f2；
F2. show that current student listens to the teacher condition grading Q according to mood analysis strategyij, it is to revere [0.9~1], meet respectively [0.8~0.9], tranquil [0.6~0.8], low [0.4~0.6], sad [0.2~0.4] and detest [0~0.2]；
F3. the behavior of classroom interactions is identified, which has mutual-action behavior, and then bonus point 1 is divided；Otherwise bonus point is denoted as 0 point；
F4. the classroom atmosphere scoring of this class is evaluated:
In formula, QijIt listens to the teacher condition grading for j-th of student of current i-th；uijFor j-th of student's bonus point of current i-th Scoring；M is the total number of persons attended class；N is the number of this class assessment；wijIt listens to the teacher shared by condition grading originally for j-th of student of i-th The percentage of secondary scoring；X is this classroom atmosphere appraisal result.
Include: in above-mentioned steps c
C1. Mask-RCNN model divided ownership facial image is used；
C2. image information is handled, using whitening approach to the amplitude normalization on each feature axis of data；
C3. by the picture input Xception in whole classroom, the characteristic value of all faces in classroom is extracted；
C4. it is generated with FPN framework and suggests that window, every image can generate N number of suggestion window, wherein N is the student people that attends class Number；
C5. suggestion window is mapped on the last layer convolution feature map of Xception；
C6.RoIAlign layers use bilinear interpolation method, and RoIAlign is generated fixed-size using each RoI feature map；
C7. full connection method is used, marks the human face region posting of face, the facial image of divided ownership, and press Sequence is numbered；
C8. front face is identified by number, training Mask-RCNN model establishes the threshold point of identification expression, such as Fruit picture does not occur threshold point, then judges that student bows, and identifies the image of Next Serial Number, if there is threshold points, then executes step Rapid c9；
C9. human facial expression recognition is put into the feature vector of extraction in SVM classifier and linear regression model (LRM), judges to learn Raw mood classification, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest.
Current student listens to the teacher condition grading Q in above-mentioned steps f2ijCalculation method it is as follows:
The face characteristic value of extraction is as input (aj1,aj2,aj3,aj4,aj5,aj6), SVM classifier is to student's facial expression Classify, linear regression model (LRM) exports the mood probability (b of 6 students to the mood degree analyzing of student's facial expressionj1, bj2,bj3,bj4,bj5,bj6), the relationship that wherein input of linear regression model (LRM) and mood export are as follows:
bjσ=ajσ*β+δ (2)
β and δ is the parameter for adjusting straight line in formula；ajσFor the facial characteristics value of j-th of student's face extraction；bijIt is current The mood probability that j-th of student listens to the teacher under state；
Best initial weights relationship are as follows:
In formula, ajσFor the facial characteristics value of j-th of student's face extraction；wjσThe spy of facial expression is extracted for j-th of student Levy weight；
Student's mood analysis strategy based on weight model show that current student listens to the teacher condition grading Qij:
Above-mentioned classroom atmosphere scoring X includes four kinds, respectively outstanding (0.8~1), good (0.6~0.8), general (0.4 ~0.6) and it is poor (0~0.4).
The invention has the following beneficial effects:
1, the present invention extracts the feature of face, Mask-RCNN model inspection face location, with SVM points using XCeption Class device and linear regression model (LRM) obtain the mood and mood degree of face.XCeption is used from lower basic thought certainly up, First by the mixing between 1*1 convolution safely responsible treatment channel, then by the space structure inside 3*3 process of convolution image, can save A large amount of parameter amount, improves arithmetic speed, the characteristic value effect for extracting face is best.
Mask-RCNN is the model improved on the basis of faster R-CNN, and RoIlign uses bilinearity and inserts side Method can more accurate Target space position；FPN framework captures stronger semantic information from further feature figure, using from upper and Lower method connection path obtains the characteristic pattern of image, the classification so as to avoid the loss of existing treatment process information, after RoI With the side for surrounding frame branch, it is competing to reduce classification using convolution framework for the branch that the mask of a generation image segmentation is added It strives, can correctly handle the overlapping region of object.
Mask-RCNN effectively detects the position of face, and XCeption can extract the characteristic value of face, svm classifier Device and linear regression model (LRM) obtain the mood and mood degree of face respectively, effectively raise classroom atmosphere assessment effect in this way Fruit, so that teaching management person preferably be promoted to improve education scheme, improve teacher's instructional strategies, promote students ' interest of study.
2, the invention proposes a kind of analysis strategies of weight model.This system is outputed various by linear regression algorithm Mood degree value carries out to reduce the error of system evaluation using the specific gravity of student's mood shared by 6 kinds of moods to student Analysis, using the weight method of least-squares estimation, has calculated optimal weight, has passed through ratio shared by each mood and its phase Corresponding each weight multiplication is added (weight adds up to 1) again, obtains the state of listening to the teacher that current student attends class.Strategy can recognize that a variety of The additional situation of mood, improves the mood accuracy of attending class of analysis student.
3, present invention is mainly applied to aspect of imparting knowledge to students, the case where analyzing student's classroom atmosphere, into once promoting quality of instruction. The present invention mainly from the overall situation of the state of student and facial expression analysis classroom, proposes a kind of Online class atmosphere assessment System and method.Purpose is good to the management in classroom, the attraction for improving teacher's teaching, raising student in order to reinforce leader Study situation and the learning behavior that draws oneself up have great importance.
Detailed description of the invention
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art The required attached drawing of embodiment is briefly described.
Fig. 1 is present system frame diagram；
Fig. 2 is present system flow chart.
Specific embodiment
The embodiment of the present invention is described in further detail with reference to the accompanying drawing.
With reference to Fig. 1, Fig. 2, a kind of Online class atmosphere assessment system, comprising:
Video flowing acquisition module, the video flowing acquisition module teach indoor video stream data for acquiring；
Data flow processing module, the data flow processing module are schemed one by one for being intercepted to the video of acquisition Picture；
Image analysis module, the image analysis module for handling image information, obtain in image all face locations and Characteristic value is extracted, segmentation face simultaneously numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify video fluxion According to the number, mood and movement posture of middle school student, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest；
Class attendance analysis module, the class attendance analysis module are used for according to the number of student statistic rate of attendance；
Classroom atmosphere evaluation module, the classroom atmosphere evaluation module are used for according to student's mood and student's movement posture, system Bow number, classroom interactions' number and student of meter student listens to the teacher rate；
Classroom atmosphere grading module, the classroom atmosphere grading module are used to carry out Classroom Teaching according to assessment data Scoring, the assessment packet include bow number, classroom interactions' number and student of student and listen to the teacher rate；
Display module, the display module is for showing the concrete condition that class attendance situation and classroom are attended class, the letter of displaying Breath includes teach teacher, class, class's total number of persons, class period section, student attendance rate and classroom atmosphere appraisal result.
A kind of Online class atmosphere appraisal procedure, it is critical that the following steps are included:
A. classroom video stream data acquires, and acquires video flowing in classroom by camera；
B. video stream data is handled, and is intercepted image one by one to the video of acquisition；
C. Mask-RCNN model divided ownership facial image is used, all face locations in image are obtained and extracts feature Value, segmentation face simultaneously numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify video stream data middle school student Number, mood and movement posture, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest, specifically:
C1. Mask-RCNN model divided ownership facial image is used；
C2. image information is handled, using whitening approach to the amplitude normalization on each feature axis of data；
C3. by the picture input Xception in whole classroom, the characteristic value of all faces in classroom is extracted；
C4. it is generated with FPN framework and suggests that window, every image can generate N number of suggestion window, wherein N is the student people that attends class Number；
C5. suggestion window is mapped on the last layer convolution feature map of Xception；
C6.RoIAlign layers use bilinear interpolation method, and RoIAlign is generated fixed-size using each RoI feature map；
C7. full connection method is used, marks the human face region posting of face, the facial image of divided ownership, and press Sequence is numbered；
C8. front face is identified by number, training Mask-RCNN model establishes the threshold point of identification expression, such as Fruit picture does not occur threshold point, then judges that student bows, and identifies the image of Next Serial Number, if there is threshold points, shows student It does not bow in state of listening to the teacher, thens follow the steps c9；
C9. human facial expression recognition is put into the feature vector of extraction in SVM classifier and linear regression model (LRM), judges to learn Raw mood classification, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest；
D. according to number of student, the statistic rate of attendance；
E. according to student's mood and student's movement posture, bow rate, classroom interactions' number and student of statistic listens to the teacher rate；
F. interval time, according to assessment data score classroom atmosphere, the assessment packet include student bow rate, Classroom interactions' number and student listen to the teacher rate:
F1. image information is analyzed, student bows, which is divided into 0 point, and student does not occur bowing executing step f2；
F2. show that current student listens to the teacher condition grading Q according to mood analysis strategyij, it is to revere [0.9~1], meet respectively [0.8~0.9], tranquil [0.6~0.8], low [0.4~0.6], sad [0.2~0.4] and detest [0~0.2]；
F3. the behavior of classroom interactions is identified, which has mutual-action behavior, and then bonus point 1 is divided；Otherwise bonus point is denoted as 0 point；
F4. the classroom atmosphere scoring of this class is evaluated:
In formula, QijIt listens to the teacher condition grading for j-th of student of current i-th；uijFor j-th of student's bonus point of current i-th Scoring；M is the total number of persons attended class；N is the number of this class assessment；wijIt listens to the teacher shared by condition grading originally for j-th of student of i-th The percentage of secondary scoring, wijEmbody the importance of classroom interaction link, w in the present embodimentij=0.95；X is this classroom atmosphere Appraisal result.The classroom atmosphere X that scores includes four kinds, respectively outstanding (0.8~1), good (0.6~0.8), it is general (0.4~ And poor (0~0.4) 0.6).
Current student listens to the teacher condition grading Q in f2ijCalculation method it is as follows:
The Online class atmosphere assessment system extracts face feature value as input (a using XCeption framej1, aj2,aj3,aj4,aj5,aj6), SVM classifier classifies to student's facial expression, and linear regression model (LRM) is to student's facial expression Mood degree analyzing, export the mood probability (b of 6 studentsj1,bj2,bj3,bj4,bj5,bj6)。
The wherein relationship of the input of linear regression model (LRM) and mood output are as follows:
bjσ=ajσ*β+δ (2)
β and δ is the parameter for adjusting straight line in formula；ajσFor the facial characteristics value of j-th of student's face extraction；bijIt is current The mood probability that j-th of student listens to the teacher under state；
Best initial weights relationship are as follows:
In formula, ajσFor the facial characteristics value of j-th of student's face extraction；wjσThe spy of facial expression is extracted for j-th of student Levy weight；
Further, the Online class atmosphere assessment system uses student's mood analysis strategy based on weight model, should Strategy analyzes specific gravity shared by every kind of mood of student, that is, can recognize that the additional situation of a variety of moods, and ultimate analysis goes out The classroom situation that current student attends class.
Finally show that current student listens to the teacher condition grading Qij:
For having 30 students, a class to have evaluated 20 times with a class, there are many assessment results for different situations, most The assessment result of this class is obtained by formula (1) eventually.The following Tables 1 and 2 of partial picture classroom atmosphere appraisal result.
The gross score of the i-th assessment of 1 30 classmates of table (without in the case where bonus point item)
Work as wijWhen=0.95, the difference of bonus point situation, the difference for the result assessed.
The different bonus point item situation of table 2 obtains different bonus point results
Serial number Bonus point situation Assessment result
1 There is no classmate to have bonus point situation 0.59965 (general)
2 Assessment is per a classmate for once having bonus point twice 0.607567 (good)
3 Add primary point each time with a classmate 0.615483 (good)
It is analyzed by Tables 1 and 2, the case where not considering bonus point item, classroom atmosphere assessment result is general；Add when considering to have When dividing situation, the value for assessing score is increasing, and the final result for assessing classroom is more preferably ideal.In classroom atmosphere evaluation process In, classroom interactions can effectively improve Classroom Teaching, promote enthusiasm of the student on classroom.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that；It still may be used To modify to technical solution documented by previous embodiment, or some or all of the technical features are equal Replacement；And these are modified or replaceed, the model for technical solution of the embodiment of the present invention that it does not separate the essence of the corresponding technical solution It encloses, should all cover within the scope of the claims and the description of the invention.

Claims (5)

1. a kind of Online class atmosphere assessment system characterized by comprising
Video flowing acquisition module, the video flowing acquisition module teach indoor video stream data for acquiring；
Data flow processing module, the data flow processing module is for being intercepted image one by one to the video of acquisition；
Image analysis module, the image analysis module obtain all face locations and extraction in image for handling image information Characteristic value, segmentation face simultaneously numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify in video stream data Number, mood and the movement posture of student, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest；
Class attendance analysis module, the class attendance analysis module are used for according to the number of student statistic rate of attendance；
Classroom atmosphere evaluation module, the classroom atmosphere evaluation module are used for according to student's mood and student's movement posture, statistics Raw bow number, classroom interactions' number and student listen to the teacher rate；
Classroom atmosphere grading module, the classroom atmosphere grading module are used to comment Classroom Teaching according to assessment data Point, the assessment packet includes bow number, classroom interactions' number and student of student and listens to the teacher rate；
Display module, the display module is for showing the concrete condition that class attendance situation and classroom are attended class, the packet of displaying Include the teacher that teaches, class, class's total number of persons, class period section, student attendance rate and classroom atmosphere appraisal result.
2. a kind of Online class atmosphere appraisal procedure, which comprises the following steps:
A. classroom video stream data acquires, and acquires video flowing in classroom by camera；
B. video stream data is handled, and is intercepted image one by one to the video of acquisition；
C. Mask-RCNN model divided ownership facial image is used, all face locations in image are obtained and extracts characteristic value, point It cuts face and numbers in order, while characteristic value is assigned to the face accordingly numbered, to identify the people of video stream data middle school student Number, mood and movement posture, middle school student's mood include revere, meet, is tranquil, low, sad and detest；
D. according to number of student, the statistic rate of attendance；
E. according to student's mood and student's movement posture, bow rate, classroom interactions' number and student of statistic listens to the teacher rate；
F. interval time scores to classroom atmosphere according to assessment data, and the assessment packet includes student and bows rate, teachers and students Interaction number and student listen to the teacher rate:
F1. image information is analyzed, student bows, which is divided into 0 point, and bowing, which does not occur, in student executes step f2；
F2. show that current student listens to the teacher condition grading Q according to mood analysis strategyij, it is to revere [0.9~1], meet [0.8 respectively ~0.9], tranquil [0.6~0.8], low [0.4~0.6], sad [0.2~0.4] and detest [0~0.2]；
F3. the behavior of classroom interactions is identified, which has mutual-action behavior, and then bonus point 1 is divided；Otherwise bonus point is denoted as 0 point；
F4. the classroom atmosphere scoring of this class is evaluated:
In formula, QijIt listens to the teacher condition grading for j-th of student of current i-th；uijFor the scoring of j-th of student's bonus point of current i-th； M is the total number of persons attended class；N is the number of this class assessment；wijListening to the teacher shared by condition grading for j-th of student of i-th, this is commented The percentage divided；X is this classroom atmosphere appraisal result.
3. Online class atmosphere according to claim 2 comments method, which is characterized in that step c includes:
C1. Mask-RCNN model divided ownership facial image is used；
C2. image information is handled, using whitening approach to the amplitude normalization on each feature axis of data；
C3. by the picture input Xception in whole classroom, the characteristic value of all faces of classroom atmosphere is extracted；
C4. it is generated with FPN framework and suggests that window, every image can generate N number of suggestion window, wherein N is number of student of attending class；
C5. suggestion window is mapped on the last layer convolution feature map of Xception；
C6.RoIAlign layers use bilinear interpolation method, and RoIAlign is generated fixed-size using each RoI feature map；
C7. full connection method is used, marks the human face region posting of face, the facial image of divided ownership, and in order It is numbered；
C8. front face is identified by number, training Mask-RCNN model establishes the threshold point of identification expression, if drawn Face does not occur threshold point, then judges that student bows, and identifies the image of Next Serial Number, if there is threshold points, thens follow the steps c9；
C9. human facial expression recognition is put into the feature vector of extraction in SVM classifier and linear regression model (LRM), judges student's Mood classification, middle school student's mood include revere, meet, is tranquil, low, it is sad and detest.
4. Online class atmosphere according to claim 3 comments method, it is characterised in that: current student listens to the teacher shape in step f2 State scoring QijCalculation method it is as follows:
The face characteristic value of extraction is as input (aj1,aj2,aj3,aj4,aj5,aj6), SVM classifier carries out student's facial expression Classification, linear regression model (LRM) export the mood probability (b of 6 students to the mood degree analyzing of student's facial expressionj1,bj2, bj3,bj4,bj5,bj6), the relationship that wherein input of linear regression model (LRM) and mood export are as follows:
bjσ=ajσ*β+δ (2)
β and δ is the parameter for adjusting straight line in formula；ajσFor the facial characteristics value of j-th of student's face extraction；bijFor under current state The mood probability that j-th of student listens to the teacher；
Best initial weights relationship are as follows:
In formula, ajσFor the facial characteristics value of j-th of student's face extraction；wjσThe feature power of facial expression is extracted for j-th of student Value；
Student's mood analysis strategy based on weight model show that current student listens to the teacher condition grading Qij:
5. Online class atmosphere appraisal procedure according to claim 4, which is characterized in that classroom atmosphere scoring X includes four Kind, it is respectively outstanding (0.8~1), good (0.6~0.8), general (0.4~0.6) and poor (0~0.4).
CN201810822970.8A 2018-07-25 2018-07-25 A kind of Online class atmosphere assessment system and method Pending CN109035089A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810822970.8A CN109035089A (en) 2018-07-25 2018-07-25 A kind of Online class atmosphere assessment system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810822970.8A CN109035089A (en) 2018-07-25 2018-07-25 A kind of Online class atmosphere assessment system and method

Publications (1)

Publication Number Publication Date
CN109035089A true CN109035089A (en) 2018-12-18

Family

ID=64645799

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810822970.8A Pending CN109035089A (en) 2018-07-25 2018-07-25 A kind of Online class atmosphere assessment system and method

Country Status (1)

Country Link
CN (1) CN109035089A (en)

Cited By (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109688384A (en) * 2019-01-08 2019-04-26 南京理工大学 A kind of teaching in classroom situation monitoring system and method based on image recognition
CN109800663A (en) * 2018-12-28 2019-05-24 华中科技大学鄂州工业技术研究院 Teachers ' teaching appraisal procedure and equipment based on voice and video feature
CN109858809A (en) * 2019-01-31 2019-06-07 浙江传媒学院 Learning quality appraisal procedure and system based on the analysis of classroom students ' behavior
CN109872587A (en) * 2019-01-07 2019-06-11 北京汉博信息技术有限公司 The processing system of multidimensional teaching data
CN110033400A (en) * 2019-03-26 2019-07-19 深圳先进技术研究院 A kind of classroom monitoring analysis system
CN110135257A (en) * 2019-04-12 2019-08-16 深圳壹账通智能科技有限公司 Business recommended data generation, device, computer equipment and storage medium
CN110427977A (en) * 2019-07-10 2019-11-08 上海交通大学 A kind of detection method of class interaction
CN110443226A (en) * 2019-08-16 2019-11-12 重庆大学 A kind of student's method for evaluating state and system based on gesture recognition
CN110458069A (en) * 2019-08-02 2019-11-15 深圳市华方信息产业有限公司 A kind of method and system based on face recognition Added Management user's on-line study state
CN110598632A (en) * 2019-09-12 2019-12-20 深圳市商汤科技有限公司 Target object monitoring method and device, electronic equipment and storage medium
CN110618757A (en) * 2019-09-23 2019-12-27 北京大米科技有限公司 Online teaching control method and device and electronic equipment
CN110619267A (en) * 2019-08-02 2019-12-27 河南科技学院 Student expression recognition method and system based on deep learning
CN110674675A (en) * 2019-08-02 2020-01-10 杭州电子科技大学 Pedestrian face anti-fraud method
CN110675669A (en) * 2019-11-01 2020-01-10 广州云蝶科技有限公司 Lesson recording method
CN110729049A (en) * 2019-11-01 2020-01-24 广州云蝶科技有限公司 Mental health early warning method
CN110808066A (en) * 2019-11-01 2020-02-18 广州云蝶科技有限公司 Teaching environment safety analysis method
CN111027865A (en) * 2019-12-12 2020-04-17 山东大学 Classroom teaching analysis and quality assessment system and method based on intelligent behavior and expression recognition
CN111105171A (en) * 2019-12-31 2020-05-05 江苏德尔斐数字科技有限公司 Method for classroom quality evaluation and rating of lecturer
CN111144255A (en) * 2019-12-18 2020-05-12 华中科技大学鄂州工业技术研究院 Method and device for analyzing non-language behaviors of teacher
CN111325645A (en) * 2020-03-18 2020-06-23 温州洪启信息科技有限公司 Teaching comprehensive analysis system based on education big data
CN111339809A (en) * 2018-12-20 2020-06-26 深圳市鸿合创新信息技术有限责任公司 Classroom behavior analysis method and device and electronic equipment
CN111460266A (en) * 2020-06-19 2020-07-28 北京和气聚力教育科技有限公司 Intelligent selection system based on cloud computing
CN111507227A (en) * 2020-04-10 2020-08-07 南京汉韬科技有限公司 Multi-student individual segmentation and state autonomous identification method based on deep learning
CN111507555A (en) * 2019-11-05 2020-08-07 浙江大华技术股份有限公司 Human body state detection method, classroom teaching quality evaluation method and related device
CN111507873A (en) * 2020-04-14 2020-08-07 四川聚阳科技集团有限公司 Classroom participation degree evaluation method based on sound and image joint sampling
CN111563697A (en) * 2020-05-21 2020-08-21 上海复岸网络信息科技有限公司 Online classroom student emotion analysis method and system
CN111563452A (en) * 2020-05-06 2020-08-21 南京师范大学镇江创新发展研究院 Multi-human body posture detection and state discrimination method based on example segmentation
CN111639565A (en) * 2020-05-19 2020-09-08 重庆大学 Audio and video combined classroom quality comprehensive evaluation method
CN111833861A (en) * 2019-04-19 2020-10-27 微软技术许可有限责任公司 Artificial intelligence based event evaluation report generation
CN111954015A (en) * 2020-08-13 2020-11-17 南京百家云科技有限公司 Method and device for realizing online live broadcast classroom
CN111985364A (en) * 2020-08-06 2020-11-24 深圳拔越软件有限公司 Remote internet education student behavior evaluation method and system
CN112070865A (en) * 2020-08-25 2020-12-11 北京大米科技有限公司 Classroom interaction method and device, storage medium and electronic equipment
CN112185195A (en) * 2020-10-28 2021-01-05 上海赛连信息科技有限公司 Method and device for controlling remote teaching classroom by AI (Artificial Intelligence)
CN112381448A (en) * 2020-11-30 2021-02-19 深圳云天励飞技术股份有限公司 Teaching quality assessment method and device based on human face space-time characteristics and electronic equipment
CN112418572A (en) * 2019-08-20 2021-02-26 成都易腾创想智能科技有限公司 Conference quality assessment system and method based on expression analysis technology
CN112598550A (en) * 2020-12-24 2021-04-02 苏州大学 Student activity multidimensional management system and management method based on behavior analysis
CN112687133A (en) * 2020-12-30 2021-04-20 湖北盟道信息科技有限公司 Audio adjusting method and system for online classroom
CN113128534A (en) * 2019-12-31 2021-07-16 北京中关村科金技术有限公司 Method, device and storage medium for emotion recognition
US11544931B2 (en) * 2020-05-26 2023-01-03 Otis Elevator Company Machine learning based human activity detection and classification in first and third person videos
US11561637B2 (en) 2020-05-20 2023-01-24 Boe Technology Group Co., Ltd. Method and device of drawing handwriting track, electronic apparatus, medium, and program product

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107316261A (en) * 2017-07-10 2017-11-03 湖北科技学院 A kind of Evaluation System for Teaching Quality based on human face analysis
CN107895244A (en) * 2017-12-26 2018-04-10 重庆大争科技有限公司 Classroom teaching quality assessment method
CN108108903A (en) * 2017-12-26 2018-06-01 重庆大争科技有限公司 Classroom teaching quality assessment system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107316261A (en) * 2017-07-10 2017-11-03 湖北科技学院 A kind of Evaluation System for Teaching Quality based on human face analysis
CN107895244A (en) * 2017-12-26 2018-04-10 重庆大争科技有限公司 Classroom teaching quality assessment method
CN108108903A (en) * 2017-12-26 2018-06-01 重庆大争科技有限公司 Classroom teaching quality assessment system

Cited By (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339809A (en) * 2018-12-20 2020-06-26 深圳市鸿合创新信息技术有限责任公司 Classroom behavior analysis method and device and electronic equipment
CN109800663A (en) * 2018-12-28 2019-05-24 华中科技大学鄂州工业技术研究院 Teachers ' teaching appraisal procedure and equipment based on voice and video feature
CN109872587A (en) * 2019-01-07 2019-06-11 北京汉博信息技术有限公司 The processing system of multidimensional teaching data
CN109688384A (en) * 2019-01-08 2019-04-26 南京理工大学 A kind of teaching in classroom situation monitoring system and method based on image recognition
CN109858809A (en) * 2019-01-31 2019-06-07 浙江传媒学院 Learning quality appraisal procedure and system based on the analysis of classroom students ' behavior
CN110033400A (en) * 2019-03-26 2019-07-19 深圳先进技术研究院 A kind of classroom monitoring analysis system
CN110135257A (en) * 2019-04-12 2019-08-16 深圳壹账通智能科技有限公司 Business recommended data generation, device, computer equipment and storage medium
CN111833861A (en) * 2019-04-19 2020-10-27 微软技术许可有限责任公司 Artificial intelligence based event evaluation report generation
CN110427977A (en) * 2019-07-10 2019-11-08 上海交通大学 A kind of detection method of class interaction
CN110458069A (en) * 2019-08-02 2019-11-15 深圳市华方信息产业有限公司 A kind of method and system based on face recognition Added Management user's on-line study state
CN110619267A (en) * 2019-08-02 2019-12-27 河南科技学院 Student expression recognition method and system based on deep learning
CN110674675A (en) * 2019-08-02 2020-01-10 杭州电子科技大学 Pedestrian face anti-fraud method
CN110619267B (en) * 2019-08-02 2022-12-16 河南科技学院 Deep learning-based student expression recognition method and system
CN110443226B (en) * 2019-08-16 2022-01-25 重庆大学 Student state evaluation method and system based on posture recognition
CN110443226A (en) * 2019-08-16 2019-11-12 重庆大学 A kind of student's method for evaluating state and system based on gesture recognition
CN112418572A (en) * 2019-08-20 2021-02-26 成都易腾创想智能科技有限公司 Conference quality assessment system and method based on expression analysis technology
CN110598632A (en) * 2019-09-12 2019-12-20 深圳市商汤科技有限公司 Target object monitoring method and device, electronic equipment and storage medium
CN110598632B (en) * 2019-09-12 2022-09-09 深圳市商汤科技有限公司 Target object monitoring method and device, electronic equipment and storage medium
CN110618757A (en) * 2019-09-23 2019-12-27 北京大米科技有限公司 Online teaching control method and device and electronic equipment
CN110675669A (en) * 2019-11-01 2020-01-10 广州云蝶科技有限公司 Lesson recording method
CN110729049B (en) * 2019-11-01 2023-09-22 广州云蝶科技有限公司 Psychological health early warning method
CN110729049A (en) * 2019-11-01 2020-01-24 广州云蝶科技有限公司 Mental health early warning method
CN110808066A (en) * 2019-11-01 2020-02-18 广州云蝶科技有限公司 Teaching environment safety analysis method
CN111507555A (en) * 2019-11-05 2020-08-07 浙江大华技术股份有限公司 Human body state detection method, classroom teaching quality evaluation method and related device
CN111507555B (en) * 2019-11-05 2023-11-14 浙江大华技术股份有限公司 Human body state detection method, classroom teaching quality evaluation method and related device
CN111027865B (en) * 2019-12-12 2024-04-02 山东大学 Teaching analysis and quality assessment system and method based on behavior and expression recognition
CN111027865A (en) * 2019-12-12 2020-04-17 山东大学 Classroom teaching analysis and quality assessment system and method based on intelligent behavior and expression recognition
CN111144255B (en) * 2019-12-18 2024-04-19 华中科技大学鄂州工业技术研究院 Analysis method and device for non-language behaviors of teacher
CN111144255A (en) * 2019-12-18 2020-05-12 华中科技大学鄂州工业技术研究院 Method and device for analyzing non-language behaviors of teacher
CN111105171A (en) * 2019-12-31 2020-05-05 江苏德尔斐数字科技有限公司 Method for classroom quality evaluation and rating of lecturer
CN113128534A (en) * 2019-12-31 2021-07-16 北京中关村科金技术有限公司 Method, device and storage medium for emotion recognition
CN111325645A (en) * 2020-03-18 2020-06-23 温州洪启信息科技有限公司 Teaching comprehensive analysis system based on education big data
CN111507227B (en) * 2020-04-10 2023-04-18 南京汉韬科技有限公司 Multi-student individual segmentation and state autonomous identification method based on deep learning
CN111507227A (en) * 2020-04-10 2020-08-07 南京汉韬科技有限公司 Multi-student individual segmentation and state autonomous identification method based on deep learning
CN111507873A (en) * 2020-04-14 2020-08-07 四川聚阳科技集团有限公司 Classroom participation degree evaluation method based on sound and image joint sampling
CN111563452A (en) * 2020-05-06 2020-08-21 南京师范大学镇江创新发展研究院 Multi-human body posture detection and state discrimination method based on example segmentation
CN111563452B (en) * 2020-05-06 2023-04-21 南京师范大学镇江创新发展研究院 Multi-human-body gesture detection and state discrimination method based on instance segmentation
CN111639565A (en) * 2020-05-19 2020-09-08 重庆大学 Audio and video combined classroom quality comprehensive evaluation method
US11561637B2 (en) 2020-05-20 2023-01-24 Boe Technology Group Co., Ltd. Method and device of drawing handwriting track, electronic apparatus, medium, and program product
CN111563697A (en) * 2020-05-21 2020-08-21 上海复岸网络信息科技有限公司 Online classroom student emotion analysis method and system
US11544931B2 (en) * 2020-05-26 2023-01-03 Otis Elevator Company Machine learning based human activity detection and classification in first and third person videos
CN111460266A (en) * 2020-06-19 2020-07-28 北京和气聚力教育科技有限公司 Intelligent selection system based on cloud computing
CN111985364A (en) * 2020-08-06 2020-11-24 深圳拔越软件有限公司 Remote internet education student behavior evaluation method and system
CN111954015A (en) * 2020-08-13 2020-11-17 南京百家云科技有限公司 Method and device for realizing online live broadcast classroom
CN111954015B (en) * 2020-08-13 2021-07-20 南京百家云科技有限公司 Method and device for realizing online live broadcast classroom
CN112070865A (en) * 2020-08-25 2020-12-11 北京大米科技有限公司 Classroom interaction method and device, storage medium and electronic equipment
CN112185195A (en) * 2020-10-28 2021-01-05 上海赛连信息科技有限公司 Method and device for controlling remote teaching classroom by AI (Artificial Intelligence)
CN112381448A (en) * 2020-11-30 2021-02-19 深圳云天励飞技术股份有限公司 Teaching quality assessment method and device based on human face space-time characteristics and electronic equipment
CN112381448B (en) * 2020-11-30 2023-10-13 深圳云天励飞技术股份有限公司 Teaching quality assessment method and device based on face space-time characteristics and electronic equipment
CN112598550A (en) * 2020-12-24 2021-04-02 苏州大学 Student activity multidimensional management system and management method based on behavior analysis
CN112598550B (en) * 2020-12-24 2024-03-26 苏州大学 Student activity multidimensional management system and method based on behavior analysis
CN112687133A (en) * 2020-12-30 2021-04-20 湖北盟道信息科技有限公司 Audio adjusting method and system for online classroom

Similar Documents

Publication Publication Date Title
CN109035089A (en) A kind of Online class atmosphere assessment system and method
CN107316261A (en) A kind of Evaluation System for Teaching Quality based on human face analysis
CN111709358B (en) Teacher-student behavior analysis system based on classroom video
CN108648757A (en) A kind of analysis method based on various dimensions Classroom Information
CN107491890A (en) One kind can quantify Classroom Teaching Quality Assessment system and method
CN106851216A (en) A kind of classroom behavior monitoring system and method based on face and speech recognition
CN108171414A (en) Evaluation System for Teaching Quality
CN111291613B (en) Classroom performance evaluation method and system
CN108898115B (en) Data processing method, storage medium and electronic device
CN109255028A (en) Quality of instruction integrated evaluating method based on teaching evaluation data reliability
Baker et al. Extending log-based affect detection to a multi-user virtual environment for science
CN110135282A (en) A kind of examinee based on depth convolutional neural networks model later plagiarizes cheat detection method
CN115239527A (en) Teaching behavior analysis system for teaching characteristic fusion and modeling based on knowledge base
CN110363129A (en) Autism early screening system based on smile normal form and audio-video behavioural analysis
CN111507227A (en) Multi-student individual segmentation and state autonomous identification method based on deep learning
Ma et al. A deep learning approach for online learning emotion recognition
Kim et al. The impact of image and voice with pedagogical agents
CN111523445B (en) Examination behavior detection method based on improved Openpost model and facial micro-expression
CN109447863A (en) A kind of 4MAT real-time analysis method and system
CN115409413A (en) Multi-source data management method, system and storage medium for student teaching and research
CN114255426A (en) Student concentration degree evaluation system based on video recognition and voice separation technology
CN115188051A (en) Object behavior-based online course recommendation method and system
CN111950472A (en) Teacher grinding evaluation method and system
CN107909285A (en) A kind of method for student's cooperative learning intelligent packet
CN112750057A (en) Student learning behavior database establishing, analyzing and processing method based on big data and cloud computing and cloud data platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20181218