CN106851216B - A kind of classroom behavior monitoring system and method based on face and speech recognition - Google Patents
A kind of classroom behavior monitoring system and method based on face and speech recognition Download PDFInfo
- Publication number
- CN106851216B CN106851216B CN201710142279.0A CN201710142279A CN106851216B CN 106851216 B CN106851216 B CN 106851216B CN 201710142279 A CN201710142279 A CN 201710142279A CN 106851216 B CN106851216 B CN 106851216B
- Authority
- CN
- China
- Prior art keywords
- teacher
- student
- classroom
- facial expression
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012544 monitoring process Methods 0.000 title claims abstract description 22
- 230000006399 behavior Effects 0.000 claims abstract description 86
- 230000008921 facial expression Effects 0.000 claims abstract description 60
- 230000003542 behavioural effect Effects 0.000 claims abstract description 25
- 230000000694 effects Effects 0.000 claims abstract description 17
- 239000000284 extract Substances 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims description 23
- 210000004709 eyebrow Anatomy 0.000 claims description 12
- 241001633942 Dais Species 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 210000003128 head Anatomy 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 5
- 241000746998 Tragus Species 0.000 claims description 3
- 238000000691 measurement method Methods 0.000 claims description 3
- 230000001815 facial effect Effects 0.000 claims 2
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000011156 evaluation Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013019 agitation Methods 0.000 description 1
- 238000003556 assay Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 210000004209 hair Anatomy 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The classroom behavior monitoring system and method based on face and speech recognition that the invention discloses a kind of, comprising the following steps: the video information of classroom middle school student, teacher are acquired by camera；The voice messaging of classroom middle school student, teacher are acquired by sound pick-up outfit；Main control processor pre-processes the video information of the student, teacher that receive, extracts the facial expression feature and behavioural characteristic of student, teacher；Main control processor handles the voice messaging of the student received, extracts student's phonetic feature；Main control processor handles the speech data information of the teacher received, extracts teacher's phonetic feature, calculates the score value of teachers ' teaching effect, makes evaluation to teachers ' teaching according to score and provide guidance instruction.The present invention is observed by the classroom behavior to teacher, student in classroom, improves the accuracy and objectivity of evaluation, can be improved teaching method and be promoted quality of instruction.
Description
Technical field
The classroom behavior monitoring system and method based on face and speech recognition that the invention mainly relates to a kind of.
Background technique
Classroom behavior is monitored, is the important link of school's evaluation quality of instruction, fully understands the level of teaching of teacher
The reaction attended class with student just can guarantee high quality teaching level.It is existing, for classroom behavior monitoring using student record or
Teacher's person test simulation, teacher observe the mode of supervision, and such mode can not give full play to the learning interest of student, Bu Nengping
The teaching efficiency of valence teacher not can be implemented simultaneously acquisition, analysis, record and the evaluation of the classroom behavior to student, teacher.Cause
How this, accurately sample the classroom behavior of student and teacher using based on face recognition technology and speech recognition technology
With intellectual analysis and evaluation, realization is observed and is recorded to the expression behaviour of teaching classroom middle school student and teacher, effectively improves
Classroom teaching effect, promoting the development in an all-round way of student is to be worth the project of research and development.
Summary of the invention
In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a kind of classroom row based on face and speech recognition
For monitoring system and method, data collection and assay are carried out in terms of assessment of students' behavior and teacher's behaviors evaluate two,
The feedbacks such as performance and teacher praise and criticism on classroom of the student on classroom are collected, and are analyzed by data, evaluation is learned
Raw learning state and learning effect, while teachers ' teaching effect is evaluated.
The technical scheme adopted by the invention is that:
A kind of classroom behavior monitoring system based on face and speech recognition, including
Video Collection System, including be mounted on four corners in classroom Omnidirection rotary pick-up head, with camera phase
Decoder even, the image splitter being connected with decoder, for acquiring the video data information of classroom middle school student, teacher；
Voice messaging acquisition system, including the sound pick-up outfit being mounted below student's desk and on dais, for acquiring
The speech data information of classroom middle school student, teacher；
Main control processor pre-processes the video data information and voice messaging of student, teacher, extracts student, religion
The facial expression feature and behavioural characteristic of teacher；
Student, the facial expression feature of teacher and behavioural characteristic are compared in standard form, and count by analysis processor
Calculate phase reserved portion.
A kind of classroom behavior monitoring method based on face and speech recognition, comprising the following steps:
Step 1: by the camera that is mounted on four, classroom corner acquire classroom middle school student, teacher video counts it is believed that
Breath；The voice data letter of classroom middle school student, teacher are acquired by being mounted on the sound pick-up outfit below student's desk and on dais
Breath；
Step 2: collected student, teacher video data information through decoder decode and image splitter segmentation after,
It is sent to main control processor；The speech data information of collected student, teacher are sent to main control processor；
Step 3: main control processor pre-processes the video data information of the student received, the face of student is extracted
Portion's expressive features and behavioural characteristic, by analysis processor by student's facial expression feature and student's facial expression standard form into
Row compares, and calculates classroom middle school student facial expression according to comparison result and shows score, by students ' behavior feature and students ' behavior mark
Quasi-mode plate is compared, and calculates classroom middle school student behavior expression score according to comparison result；
Step 4: main control processor pre-processes the video data information of the teacher received, the face of teacher is extracted
Portion's expressive features and behavioural characteristic, by analysis processor by teacher's facial expression feature with and teacher's facial expression standard form
It is compared, calculates teacher to the score of student classroom performance facial expression reaction, by teacher's behaviors feature according to comparison result
It is compared with teacher's behaviors standard form, calculates the score that teacher reacts student classroom expression behaviour according to comparison result；
Step 5: main control processor handles the speech data information of the student received, it is special to extract student's voice
Sign, training token sound template, student's phonetic feature is compared with token sound template, calculates classroom according to comparison result
In every student speech number and speech ratio when frequency, time limit of speech length and panel discussion；
Step 6: main control processor handles the speech data information of the teacher received, it is special to extract teacher's voice
Sign, calculates the score value of teachers ' teaching effect, and is compared with teachers ' teaching effect mean scores, when the score value is taught less than teacher
When learning effect mean scores, prompt is issued；
Step 7: main control processor is by classroom middle school student's facial expression and behavior expression score and teacher to student classroom
The score of the reaction of performance facial expression and behavior reaction is integrated, and obtains the classroom behavior total score of each student, and this is total
Divide and be compared with the student classroom behavior average mark being arranged in master controller, when the total score is average less than student classroom behavior
Score issues prompt；
Step 8: by student, the video data information of teacher, speech data information, each student classroom behavior total score
It stores with the score value of teachers ' teaching effect into the database of main control processor.
Further, the student, teacher video data information include student, the behavioural information of teacher and student, religion
The facial expression information of teacher.
Further, in the step 3, student's facial expression feature template include iris center, inner eye corner point,
External eyes angle point, nostril point, tragus point, subaurale, bicker point, crown point, is put in eyebrow and eyebrow exterior point prenasale；Student's row
Being characterized template includes raising one's hand, bowing and take notes and new line is listened to the teacher.
Further, in the step 3, the facial expression feature and behavioural characteristic and standard form for carrying out student into
Before row compares, first student's facial expression standard form and students ' behavior standard form assign and be divided, student's face table
Absorbed, happiness expression is set as 10 points in feelings standard form, and cold and detached expression is set as 4 points, and agitation is set as 1 point；Students ' behavior
Raise one's hand in standard form, bow take notes, the behavior listened to the teacher that comes back is set as 10 points, and be set as 0 point.
Further, in the step 4, teacher's facial expression standard form includes corners of the mouth radian, is put in eyebrow and eyebrow
Exterior point；The teacher's behaviors standard form includes that teacher nods number.
Further, in the step 4, the facial expression feature and behavioural characteristic and standard form for carrying out teacher into
Before row compares, first teacher's facial expression standard form and teacher's behaviors standard form assign and be divided, teacher's face table
Pleasant expression is set as 10 points in feelings standard form, and discontented expression is set as 0 point, remaining is assigned in intermediate range according to satisfaction
Point, in the teacher's behaviors standard form according to teacher nod number carry out assign point.
Further, in the step 5, student's phonetic feature include every class of student speech number and frequency, every time
Time limit of speech length and group make a speech ratio when talking about.
Further, the training token sound template in the step 5 method particularly includes: according to the instruction of each speaker
Practice speech samples and establishes the token sound template of each speaker through feature extraction.
Further, in shown step 6, the score value of teachers ' teaching effect is calculated method particularly includes: according to token sound
The voice messaging of teacher is divided into several subsections by template, its measure value is calculated using diversification meas urement method, according to
Analysis processor Plays sound bank establishes speech assessment model, and the measure value of acquisition is converted into scientificity teaching efficiency
Score value.
Compared with prior art, the beneficial effects of the present invention are:
(1) science carried out to the classroom behavior of classroom middle school student, teacher using camera and sound pick-up outfit, objectively adopted
Sample provides quantitative analysis data；Collected data are analyzed and processed by main control processor and analysis processor, are obtained
The study of student puts into degree, and evaluates its learning effect, available strategy can be taken correctly to draw based on the analysis result school
It leads student actively to express, effectively improves classroom teaching effect, promote the development in an all-round way of student；
(2) facial expression feature of student and behavioural characteristic are compared by analysis processor with standard form and are beaten
Point, instead of the mode of traditional classroom observation, improve the accuracy and objectivity of evaluation；
It is (3) of the invention by being analyzed and evaluated to students ' behavior collected on classroom and classroom interactions' behavior,
Teaching method can be improved and promote quality of instruction.
Detailed description of the invention
The accompanying drawings constituting a part of this application is used to provide further understanding of the present application, and the application's shows
Meaning property embodiment and its explanation are not constituted an undue limitation on the present application for explaining the application.
Fig. 1 is the flow chart of the classroom behavior monitoring method based on face and speech recognition in the embodiment of the present invention；
Fig. 2 is that student's voice messaging collects process flow diagram；
Fig. 3 is that teacher's voice messaging collects process flow diagram；
Fig. 4 is the structural block diagram of the classroom behavior monitoring system based on face and speech recognition in the embodiment of the present invention.
Specific embodiment
It is noted that following detailed description is all illustrative, it is intended to provide further instruction to the application.Unless another
It indicates, all technical and scientific terms used herein has usual with the application person of an ordinary skill in the technical field
The identical meanings of understanding.
It should be noted that term used herein above is merely to describe specific embodiment, and be not intended to restricted root
According to the illustrative embodiments of the application.As used herein, unless the context clearly indicates otherwise, otherwise singular
Also it is intended to include plural form, additionally, it should be understood that, when in the present specification using term "comprising" and/or " packet
Include " when, indicate existing characteristics, step, operation, device, component and/or their combination.
In a kind of typical embodiment of the application, as shown in Figure 1-3, a kind of classroom based on face and speech recognition
Behavior monitoring method, comprising the following steps:
Camera by being mounted on four, classroom corner acquires the video data information of classroom middle school student, teacher, described
Student, teacher video data information include the facial expression information of student, the behavioural information of teacher and student, teacher；Pass through
It is mounted on the speech data information that the sound pick-up outfit below student's desk and on dais acquires classroom middle school student, teacher；
The video data information of collected student is sent at master control after decoder decoding and image splitter segmentation
Manage device；Main control processor pre-processes the video data information of the student received, then carry out human facial expression recognition and
Activity recognition；For human facial expression recognition, the facial expression feature that face characteristic difference extracts student is first passed through, when extraction uses
It is extracted based on student's facial expression standard form；Student's facial expression feature template includes iris center, inner eye corner
Point, prenasale, nostril point, tragus point, subaurale, bicker point, crown point, is put in eyebrow and eyebrow exterior point external eyes angle point；For behavior
The behavioural characteristic of student is first extracted in identification, and wherein behavioural characteristic extraction focuses primarily upon four limbs, head position, student's row
It include raising one's hand, bowing and take notes and new line is listened to the teacher for standard form；
Analysis comparison is carried out by facial expression feature and behavioural characteristic of the analysis processor to the student extracted, first
Student's facial expression standard form to be assigned and is divided, absorbed, happiness expression is set as 10 points in student's facial expression standard form,
Cold and detached expression is set as 4 points, and irritated expression is set as 1 point；By analysis processor by student's facial expression feature and student face
Portion's expression standard form is compared, and is calculated classroom middle school student facial expression according to comparison result and is showed score；To students ' behavior
Standard form carry out assign point, raise one's hand in students ' behavior standard form, bow take notes, the behavior listened to the teacher that comes back is set as 10 points；
Students ' behavior feature is compared with students ' behavior standard form by analysis processor, is calculated in classroom according to comparison result
Students ' behavior shows score；
The video data information of collected teacher is sent at master control after decoder decoding and image splitter segmentation
Manage device；Main control processor pre-processes the video data information of the teacher received, is based on teacher's facial expression master die
Plate extracts the facial expression feature of teacher and extracts behavioural characteristic based on teacher's behaviors standard form, wherein teacher's facial expression mark
Quasi-mode plate includes corners of the mouth radian, is put in eyebrow and eyebrow exterior point；Teacher's behaviors standard form includes that teacher nods number；
Analysis comparison is carried out by facial expression feature and behavioural characteristic of the analysis processor to the teacher extracted, first
Teacher's facial expression standard form is assigned and is divided, pleasant expression is set as 10 points in teacher's facial expression standard form, is discontented with expression
Be set as 0 point, remaining intermediate range according to satisfaction assign point, by analysis processor by teacher's facial expression feature with and
Teacher's facial expression standard form is compared, and calculates teacher to student classroom performance facial expression reaction according to comparison result
Score；Then to teacher's behaviors standard form carry out assign point, in the teacher's behaviors standard form according to teacher nod number into
Row, which is assigned, to be divided；Teacher's behaviors feature is compared with teacher's behaviors standard form by analysis processor, according to comparison result meter
Calculate the score that teacher reacts student classroom expression behaviour；
The speech data information of collected student, teacher are sent to main control processor；Main control processor is to receiving
The speech data information of student handle, student's phonetic feature is extracted based on speaker Recognition Technology, is spoken according to each
The training phonetic material of people establishes the token sound template of each speaker through feature extraction, obtains the voice mark for completing training
Student's phonetic feature is compared by quasi-mode plate with token sound template；Student's phonetic feature includes the speech of every class of student
Number and frequency, each time limit of speech length and group are made a speech ratio when talking about, and calculate every in classroom according to comparison result
Speech ratio when raw speech number and frequency, time limit of speech length and panel discussion；
Main control processor handles the speech data information of the teacher received, is based on speech recognition technology, first
Extract teacher's phonetic feature；Secondly, voice is divided using audio segmentation algorithm according to trained token sound template is completed
For several subsections, its measure value is calculated using diversification meas urement method according to different voice messagings；Finally, according to
Standards for teachers sound bank establishes speech assessment model in analysis processor, and the measure value of acquisition is converted into scientificity teaching
The score value of effect；Wherein, comprising feeding back to learner answering questions positive feedback and negative sense in standards for teachers sound bank, positive feedback is back
It answers correctly, negative sense is fed back to erroneous answers.
Student classroom septum reset expression and behavior expression score and teacher are showed face to student classroom by main control processor
The reaction of portion's expression and the score of behavior reaction are integrated, and obtain the classroom behavior total score of each student, and by the total score and master
The student classroom behavior average mark being arranged in controller is compared, when total score is less than student classroom behavior average mark, hair
It prompts out；
Student, the video data information of teacher, speech data information, the classroom behavior total score of each student and teacher are taught
The score value for learning effect is stored into the database of main control processor.
As shown in figure 4, a kind of classroom behavior monitoring system based on face and speech recognition, including
Video Collection System, including be mounted on four corners in classroom Omnidirection rotary pick-up head, with camera phase
Decoder even, the image splitter being connected with decoder, acquire the video data information of classroom middle school student, teacher；
Voice messaging acquisition system acquires classroom including the sound pick-up outfit being mounted below student's desk and on dais
The speech data information of middle school student, teacher；
Main control processor pre-processes the video data information and voice messaging of student, teacher, extracts student, religion
The facial expression feature and behavioural characteristic of teacher；
Student, the facial expression feature of teacher and behavioural characteristic are compared in standard form, and count by analysis processor
Calculate phase reserved portion.
Above-mentioned, although the foregoing specific embodiments of the present invention is described with reference to the accompanying drawings, not protects model to the present invention
The limitation enclosed, those skilled in the art should understand that, based on the technical solutions of the present invention, those skilled in the art are not
Need to make the creative labor the various modifications or changes that can be made still within protection scope of the present invention.
Claims (10)
1. a kind of classroom behavior monitoring method based on face and speech recognition, characterized in that the following steps are included:
Step 1: acquiring the video data information of classroom middle school student, teacher by the camera for being mounted on four, classroom corner；It is logical
Cross the speech data information for being mounted on the acquisition of the sound pick-up outfit below student's desk and on dais classroom middle school student, teacher；
Step 2: collected student, teacher video data information through decoder decode and image splitter segmentation after, send
To main control processor；The speech data information of collected student, teacher are sent to main control processor；
Step 3: main control processor pre-processes the video data information of the student received, the facial table of student is extracted
Feelings feature and behavioural characteristic are compared student's facial expression feature and student's facial expression standard form by analysis processor
Compared with according to comparison result calculating classroom middle school student facial expression performance score, by students ' behavior feature and students ' behavior master die
Plate is compared, and calculates classroom middle school student behavior expression score according to comparison result；
Step 4: main control processor pre-processes the video data information of the teacher received, the facial table of teacher is extracted
Feelings feature and behavioural characteristic, by analysis processor by teacher's facial expression feature with and teacher's facial expression standard form carry out
Compare, calculates teacher to the score of student classroom performance facial expression reaction, by teacher's behaviors feature and religion according to comparison result
Teacher's behavioral standard template is compared, and calculates the score that teacher reacts student classroom expression behaviour according to comparison result；
Step 5: main control processor handles the speech data information of the student received, student's phonetic feature, instruction are extracted
Practice token sound template, student's phonetic feature is compared with token sound template, is calculated according to comparison result every in classroom
Speech ratio when the speech number of position student and frequency, time limit of speech length and panel discussion；
Step 6: main control processor handles the speech data information of the teacher received, teacher's phonetic feature, meter are extracted
The score value of teachers ' teaching effect is calculated, and is compared with teachers ' teaching effect mean scores, when the score value is imitated less than teachers ' teaching
When fruit mean scores, prompt is issued；
Step 7: main control processor shows student classroom classroom middle school student's facial expression and behavior expression score and teacher
Facial expression reaction and the score of behavior reaction are integrated, and obtain the classroom behavior total score of each student, and by the total score with
The student classroom behavior average mark being arranged in master controller is compared, when the total score is less than student classroom behavior average mark
Number issues prompt；
Step 8: by student, the video data information of teacher, speech data information, each student classroom behavior total score and religion
The score value of teacher's teaching efficiency is stored into the database of main control processor.
2. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
State student, the video data information of teacher include student, the behavioural information of teacher and student, teacher facial expression information.
3. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
It states in step 3, student's facial expression standard form includes iris center, inner eye corner point, external eyes angle point, prenasale, nostril
Point, subaurale, bicker point, crown point, is put in eyebrow and eyebrow exterior point tragus point；The students ' behavior standard form includes raising one's hand, being low
Head is taken notes and new line is listened to the teacher.
4. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
It states in step 3, before the facial expression feature and behavioural characteristic for carrying out student are compared with standard form, first to student
Facial expression standard form and students ' behavior standard form, which assign, to be divided, and is absorbed in student's facial expression standard form, is high
Emerging expression is set as 10 points, and cold and detached expression is set as 4 points, and irritated expression is set as 1 point；Raise one's hand in students ' behavior standard form,
Bow take notes, the behavior listened to the teacher that comes back is set as 10 points.
5. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
It states in step 4, teacher's facial expression standard form includes corners of the mouth radian, is put in eyebrow and eyebrow exterior point；The teacher's behaviors mark
Quasi-mode plate includes that teacher nods number.
6. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
It states in step 4, before the facial expression feature and behavioural characteristic for carrying out teacher are compared with standard form, first to teacher
Facial expression standard form and teacher's behaviors standard form, which assign, to be divided, pleasant expression in teacher's facial expression standard form
It is set as 10 points, discontented expression is set as 0 point, remaining, which is assigned in intermediate range according to satisfaction, divides, the teacher's behaviors standard
In template according to teacher nod number carry out assign point.
7. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
It states in step 5, student's phonetic feature includes the speech number and frequency of every class of student, each time limit of speech length and group
It makes a speech when talking about ratio.
8. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
State the training token sound template in step 5 method particularly includes: according to the training speech samples of each speaker, mention through feature
It takes, establishes the token sound template of each speaker.
9. a kind of classroom behavior monitoring method based on face and speech recognition according to claim 1, characterized in that institute
Show in step 6, calculate the score value of teachers ' teaching effect method particularly includes: believes the voice of teacher according to token sound template
Breath is divided into several subsections, its measure value is calculated using diversification meas urement method, according to analysis processor Plays language
Speech assessment model is established in sound library, and the measure value of acquisition is converted into the score value of scientificity teaching efficiency.
10. a kind of system for realizing the classroom behavior monitoring method as described in claim 1 based on face and speech recognition,
It is characterized in, comprising:
Video Collection System, the Omnidirection rotary pick-up head, connected to the camera including being mounted on four corners in classroom
Decoder, the image splitter being connected with decoder, for acquiring the video data information of classroom middle school student, teacher；
Voice messaging acquisition system, including the sound pick-up outfit being mounted below student's desk and on dais, for acquiring classroom
The speech data information of middle school student, teacher；
Main control processor pre-processes the video data information and voice messaging of student, teacher, extracts student, teacher
Facial expression feature and behavioural characteristic；
Student, the facial expression feature of teacher and behavioural characteristic are compared in standard form, and calculate phase by analysis processor
Reserved portion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710142279.0A CN106851216B (en) | 2017-03-10 | 2017-03-10 | A kind of classroom behavior monitoring system and method based on face and speech recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710142279.0A CN106851216B (en) | 2017-03-10 | 2017-03-10 | A kind of classroom behavior monitoring system and method based on face and speech recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106851216A CN106851216A (en) | 2017-06-13 |
| CN106851216B true CN106851216B (en) | 2019-05-28 |
Family
ID=59144257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710142279.0A Active CN106851216B (en) | 2017-03-10 | 2017-03-10 | A kind of classroom behavior monitoring system and method based on face and speech recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106851216B (en) |
Families Citing this family (82)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292271B (en) * | 2017-06-23 | 2020-02-14 | 北京易真学思教育科技有限公司 | Learning monitoring method and device and electronic equipment |
| CN107085721A (en) * | 2017-06-26 | 2017-08-22 | 厦门劢联科技有限公司 | A kind of intelligence based on Identification of Images patrols class management system |
| CN107316261A (en) * | 2017-07-10 | 2017-11-03 | 湖北科技学院 | A kind of Evaluation System for Teaching Quality based on human face analysis |
| CN107341618A (en) * | 2017-07-14 | 2017-11-10 | 安徽智星交通科技股份有限公司 | Class management method and system |
| CN107248046A (en) * | 2017-08-01 | 2017-10-13 | 中州大学 | A kind of moral and political science Classroom Teaching device and method |
| CN107609736A (en) * | 2017-08-09 | 2018-01-19 | 广州思涵信息科技有限公司 | A kind of teaching diagnostic analysis system and method for integrated application artificial intelligence technology |
| CN107491890A (en) * | 2017-08-29 | 2017-12-19 | 广州思涵信息科技有限公司 | One kind can quantify Classroom Teaching Quality Assessment system and method |
| CN107609517B (en) * | 2017-09-15 | 2020-10-30 | 华中科技大学 | Classroom behavior detection system based on computer vision |
| TW201915854A (en) * | 2017-10-02 | 2019-04-16 | 曾靜怡 | Automatic categorizing system for electronic contact book automatically establishes an exclusive electronic contact book of each student end containing personal learning, life, time, personality and resume |
| CN109753855A (en) * | 2017-11-06 | 2019-05-14 | 北京易真学思教育科技有限公司 | The determination method and device of teaching scene state |
| CN108154304A (en) * | 2017-12-26 | 2018-06-12 | 重庆大争科技有限公司 | There is the server of Teaching Quality Assessment |
| CN108460700B (en) * | 2017-12-28 | 2021-11-16 | 北京科教科学研究院 | Intelligent student education management regulation and control system |
| CN108376305A (en) * | 2017-12-30 | 2018-08-07 | 中国移动通信集团福建有限公司 | Training quality appraisal procedure, device, equipment and medium |
| CN108197657B (en) * | 2018-01-04 | 2022-04-19 | 成都寻道科技有限公司 | Student economic condition prediction method based on campus data |
| CN108229855A (en) * | 2018-02-06 | 2018-06-29 | 上海小蚁科技有限公司 | Method for monitoring service quality and device, computer readable storage medium, terminal |
| CN108229441B (en) * | 2018-02-06 | 2022-03-29 | 浙江大学城市学院 | Classroom teaching automatic feedback system and feedback method based on image and voice analysis |
| CN108269576B (en) * | 2018-02-13 | 2020-12-15 | 福州外语外贸学院 | Student classroom performance scoring method and terminal |
| CN108416797A (en) * | 2018-02-27 | 2018-08-17 | 鲁东大学 | A kind of method, equipment and the storage medium of detection Behavioral change |
| CN108389475A (en) * | 2018-03-05 | 2018-08-10 | 山东星科素养教育科技有限公司 | A kind of practical training method and system of multi-user Cooperation |
| CN108182838A (en) * | 2018-03-12 | 2018-06-19 | 山西师范大学 | A kind of real-time evaluation system of Piano lesson |
| TWI684159B (en) * | 2018-03-20 | 2020-02-01 | 麥奇數位股份有限公司 | Instant monitoring method for interactive online teaching |
| CN108256512A (en) * | 2018-03-22 | 2018-07-06 | 长春大学 | Listen the raw inclusive education classroom auxiliary system of barrier and device |
| CN108470255B (en) * | 2018-04-12 | 2021-02-02 | 上海小蚁科技有限公司 | Workload statistical method and device, storage medium and computing equipment |
| CN108694679A (en) * | 2018-05-15 | 2018-10-23 | 北京中庆现代技术股份有限公司 | A kind of method student's learning state detection and precisely pushed |
| CN108875785B (en) * | 2018-05-17 | 2021-04-06 | 深圳市鹰硕技术有限公司 | Attention degree detection method and device based on behavior feature comparison |
| CN108648757B (en) * | 2018-06-14 | 2020-10-16 | 北京中庆现代技术股份有限公司 | Analysis method based on multi-dimensional classroom information |
| CN108898084A (en) * | 2018-06-20 | 2018-11-27 | 四川文理学院 | A kind of method and device thereof of robot identification student's state auxiliary classroom instruction |
| CN108921069B (en) * | 2018-06-22 | 2022-05-06 | 张小勇 | Infant interest and hobby recommendation method and system |
| CN108898523A (en) * | 2018-06-25 | 2018-11-27 | 安徽三联学院 | A kind of Education Administration Information System based on Chinese |
| CN108961115A (en) * | 2018-07-02 | 2018-12-07 | 百度在线网络技术（北京）有限公司 | Method, apparatus, equipment and the computer readable storage medium of teaching data analysis |
| CN108877771B (en) * | 2018-07-11 | 2021-01-26 | 北京大米科技有限公司 | Data processing method, storage medium and electronic device |
| CN109657529A (en) * | 2018-07-26 | 2019-04-19 | 台州学院 | Classroom teaching effect evaluation system based on human facial expression recognition |
| CN109087222A (en) * | 2018-08-01 | 2018-12-25 | 阔地教育科技有限公司 | Classroom data analysing method and system |
| CN109344682A (en) * | 2018-08-02 | 2019-02-15 | 平安科技（深圳）有限公司 | Classroom monitoring method, device, computer equipment and storage medium |
| CN108881861A (en) * | 2018-08-08 | 2018-11-23 | 湖北统讯智能科技有限公司 | A kind of students ' behavior identification and management system |
| WO2020034902A1 (en) * | 2018-08-11 | 2020-02-20 | 昆山美卓智能科技有限公司 | Smart desk having status monitoring function, monitoring system server, and monitoring method |
| CN109214664B (en) * | 2018-08-21 | 2021-04-16 | 重庆乐教科技有限公司 | Emotional behavior comprehensive analysis system based on artificial intelligence |
| CN109089099A (en) * | 2018-09-05 | 2018-12-25 | 广州维纳斯家居股份有限公司 | Image processing method, device, equipment and storage medium |
| CN109360458A (en) * | 2018-10-25 | 2019-02-19 | 重庆鲁班机器人技术研究院有限公司 | Interest assistant teaching method, device and robot |
| CN109214966A (en) * | 2018-10-25 | 2019-01-15 | 重庆鲁班机器人技术研究院有限公司 | Learning effect acquisition methods, device and electronic equipment |
| CN109657553B (en) * | 2018-11-16 | 2023-06-20 | 江苏科技大学 | Student classroom attention detection method |
| CN111339809A (en) * | 2018-12-20 | 2020-06-26 | 深圳市鸿合创新信息技术有限责任公司 | Classroom behavior analysis method and device and electronic equipment |
| CN109523441A (en) * | 2018-12-20 | 2019-03-26 | 合肥凌极西雅电子科技有限公司 | A kind of Teaching Management Method and system based on video identification |
| CN109544421A (en) * | 2018-12-20 | 2019-03-29 | 合肥凌极西雅电子科技有限公司 | A kind of intelligent tutoring management system and method based on children |
| CN109889916B (en) * | 2019-01-07 | 2023-04-07 | 北京汉博信息技术有限公司 | Application system of recorded broadcast data |
| CN109889881B (en) * | 2019-01-21 | 2021-07-06 | 广东精标科技股份有限公司 | Teacher classroom teaching data acquisition system |
| CN109919434A (en) * | 2019-01-28 | 2019-06-21 | 华中科技大学 | A kind of classroom performance intelligent Evaluation method based on deep learning |
| CN111667128B (en) * | 2019-03-05 | 2024-01-05 | 杭州海康威视系统技术有限公司 | Teaching quality assessment method, device and system |
| CN109919816A (en) * | 2019-03-21 | 2019-06-21 | 北京旷视科技有限公司 | Cource arrangement method and device, storage medium and electronic equipment based on data analysis |
| CN110033659B (en) * | 2019-04-26 | 2022-01-21 | 北京大米科技有限公司 | Remote teaching interaction method, server, terminal and system |
| CN110276246A (en) * | 2019-05-09 | 2019-09-24 | 威比网络科技(上海)有限公司 | Course index detects alarm method, device, electronic equipment, storage medium |
| CN111967703A (en) * | 2019-05-20 | 2020-11-20 | 唐德海 | Language-behavior big data synchronous analysis system for classroom teaching |
| CN110232346A (en) * | 2019-06-06 | 2019-09-13 | 南京睦泽信息科技有限公司 | A kind of video intelligent analysis system based on deep learning |
| CN112116181B (en) * | 2019-06-20 | 2022-12-02 | 北京新唐思创教育科技有限公司 | Classroom quality model training method, classroom quality evaluation method and classroom quality evaluation device |
| CN111353920A (en) * | 2019-06-28 | 2020-06-30 | 鸿合科技股份有限公司 | Intelligent blackboard |
| CN110444053B (en) * | 2019-07-04 | 2021-11-30 | 卓尔智联(武汉)研究院有限公司 | Language learning method, computer device and readable storage medium |
| CN110349063A (en) * | 2019-07-16 | 2019-10-18 | 广州市教育研究院 | A kind of school work growth curve test method and system |
| CN110600033B (en) * | 2019-08-26 | 2022-04-05 | 北京大米科技有限公司 | Learning condition evaluation method and device, storage medium and electronic equipment |
| CN110837947B (en) * | 2019-10-11 | 2022-05-31 | 杭州电子科技大学 | Assessment method for teacher teaching concentration degree based on audio and video analysis technology |
| CN111027584A (en) * | 2019-10-23 | 2020-04-17 | 宋飞 | Classroom behavior identification method and device |
| CN111507555B (en) * | 2019-11-05 | 2023-11-14 | 浙江大华技术股份有限公司 | Human body state detection method, classroom teaching quality evaluation method and related device |
| CN110992741B (en) * | 2019-11-15 | 2022-05-17 | 深圳算子科技有限公司 | Learning auxiliary method and system based on classroom emotion and behavior analysis |
| CN111028853A (en) * | 2019-11-25 | 2020-04-17 | 探客柏瑞科技（北京）有限公司 | Spoken language expressive force evaluation method and system |
| CN111027486B (en) * | 2019-12-11 | 2023-11-14 | 李思娴 | Auxiliary analysis and evaluation system and method for classroom teaching effect big data of middle and primary schools |
| CN111027865B (en) * | 2019-12-12 | 2024-04-02 | 山东大学 | Teaching analysis and quality assessment system and method based on behavior and expression recognition |
| CN111390919A (en) * | 2020-03-09 | 2020-07-10 | 合肥贤坤信息科技有限公司 | Accompany robot intelligence image recognition behavior analysis system |
| CN111523460A (en) * | 2020-04-23 | 2020-08-11 | 上海铠盾科技有限公司 | Standard operation behavior detection system |
| CN111401797B (en) * | 2020-05-09 | 2023-04-18 | 华南师范大学 | Teaching quality evaluation method and system |
| CN111882305A (en) * | 2020-05-09 | 2020-11-03 | 北京文香信息技术有限公司 | Behavior analysis device and behavior monitoring analysis system |
| CN111291840A (en) * | 2020-05-12 | 2020-06-16 | 成都派沃智通科技有限公司 | Student classroom behavior recognition system, method, medium and terminal device |
| CN111800646A (en) * | 2020-06-24 | 2020-10-20 | 北京安博盛赢教育科技有限责任公司 | Method, device, medium and electronic equipment for monitoring teaching effect |
| CN112001275A (en) * | 2020-08-09 | 2020-11-27 | 成都未至科技有限公司 | Robot for collecting student information |
| CN112132009A (en) * | 2020-09-22 | 2020-12-25 | 湖南惟楚有才教育科技有限公司 | Classroom behavior analysis method and system and electronic equipment |
| CN112287844B (en) * | 2020-10-30 | 2023-04-18 | 北京市商汤科技开发有限公司 | Student situation analysis method and device, electronic device and storage medium |
| CN112861730A (en) * | 2021-02-09 | 2021-05-28 | 北京文香信息技术有限公司 | Feedback method and device of classroom behavior, electronic equipment and storage medium |
| CN112907408A (en) * | 2021-03-01 | 2021-06-04 | 北京安博创赢教育科技有限责任公司 | Method, device, medium and electronic equipment for evaluating learning effect of students |
| CN113743250B (en) * | 2021-08-16 | 2024-02-13 | 华中师范大学 | Construction method and system of classroom teaching behavior event description model |
| CN113992932B (en) * | 2021-11-01 | 2024-03-19 | 北京高途云集教育科技有限公司 | Information prompting method, device, electronic equipment and readable storage medium |
| CN114760442B (en) * | 2022-04-06 | 2023-03-24 | 英才(广州)在线教育科技有限公司 | Monitoring system for online education management |
| CN115049970A (en) * | 2022-08-15 | 2022-09-13 | 北京师范大学 | Student classroom performance evaluation method based on multi-mode audio and video technology |
| CN115130933A (en) * | 2022-09-01 | 2022-09-30 | 广东银谷科技股份有限公司 | Teaching rating system based on quantitative indexes |
| CN115936944B (en) * | 2023-01-31 | 2023-10-13 | 西昌学院 | Virtual teaching management method and device based on artificial intelligence |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542849A (en) * | 2012-01-20 | 2012-07-04 | 东南大学 | Formative evaluation system |
| CN103295171A (en) * | 2013-06-05 | 2013-09-11 | 广州市奥威亚电子科技有限公司 | Automatic S-T teaching analysis method based on intelligent recording and broadcasting system |
| CN104598641A (en) * | 2015-02-12 | 2015-05-06 | 俞琳 | Teaching achievement analysis and statistics method based on cloud platform |
-
2017
- 2017-03-10 CN CN201710142279.0A patent/CN106851216B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542849A (en) * | 2012-01-20 | 2012-07-04 | 东南大学 | Formative evaluation system |
| CN103295171A (en) * | 2013-06-05 | 2013-09-11 | 广州市奥威亚电子科技有限公司 | Automatic S-T teaching analysis method based on intelligent recording and broadcasting system |
| CN104598641A (en) * | 2015-02-12 | 2015-05-06 | 俞琳 | Teaching achievement analysis and statistics method based on cloud platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106851216A (en) | 2017-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106851216B (en) | A kind of classroom behavior monitoring system and method based on face and speech recognition | |
| CN106878677B (en) | Student classroom mastery degree evaluation system and method based on multiple sensors | |
| CN111709358B (en) | Teacher-student behavior analysis system based on classroom video | |
| CN107480872B (en) | Online teaching evaluation system and method based on data exchange network | |
| CN110991381B (en) | Real-time classroom student status analysis and indication reminding system and method based on behavior and voice intelligent recognition | |
| CN105022929B (en) | A kind of cognition accuracy analysis method of personal traits value test | |
| Engwall | Analysis of and feedback on phonetic features in pronunciation training with a virtual teacher | |
| CN109815795A (en) | Classroom student's state analysis method and device based on face monitoring | |
| CN111027865B (en) | Teaching analysis and quality assessment system and method based on behavior and expression recognition | |
| US20070055523A1 (en) | Pronunciation training system | |
| CN111428175A (en) | Micro-expression recognition-based online course recommendation method and related equipment | |
| US20150302866A1 (en) | Speech affect analyzing and training | |
| CN107609736A (en) | A kind of teaching diagnostic analysis system and method for integrated application artificial intelligence technology | |
| CN110544481B (en) | S-T classification method and device based on voiceprint recognition and equipment terminal | |
| CN104537926B (en) | Listen barrier childrenese training accessory system and method | |
| CN109727608A (en) | A kind of ill voice appraisal procedure based on Chinese speech | |
| CN115713875A (en) | Virtual reality simulation teaching method based on psychological analysis | |
| CN110134863A (en) | The method and device that application program is recommended | |
| CN111695442A (en) | Online learning intelligent auxiliary system based on multi-mode fusion | |
| CN116487003A (en) | Rehabilitation training system for patient with language disorder | |
| CN108175426A (en) | A kind of lie detecting method that Boltzmann machine is limited based on depth recursion type condition | |
| US20220015687A1 (en) | Method for Screening Psychiatric Disorder Based On Conversation and Apparatus Therefor | |
| Xu et al. | Thoughts on application of artificial intelligence in teaching of different disciplines | |
| CN113256453A (en) | Learning state improvement management system | |
| McKechnie | Exploring the use of technology for assessment and intensive treatment of childhood apraxia of speech |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |