CN113705360A - Reminding method and reminding device for teaching assistance - Google Patents
Reminding method and reminding device for teaching assistance Download PDFInfo
- Publication number
- CN113705360A CN113705360A CN202110884553.8A CN202110884553A CN113705360A CN 113705360 A CN113705360 A CN 113705360A CN 202110884553 A CN202110884553 A CN 202110884553A CN 113705360 A CN113705360 A CN 113705360A
- Authority
- CN
- China
- Prior art keywords
- class
- personnel
- face recognition
- target frame
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 206010000117 Abnormal behaviour Diseases 0.000 claims abstract description 75
- 238000012549 training Methods 0.000 claims abstract description 52
- 238000012790 confirmation Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 15
- 230000002159 abnormal effect Effects 0.000 claims description 26
- 238000005192 partition Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention provides a reminding method and a reminding device for teaching assistance, and belongs to the field of intelligent teaching. The reminding method comprises the following steps: inputting the image data of each class video frame in a classroom into a class personnel model based on Yolov4 network training to obtain a class personnel target frame in the image data of each class video frame; correspondingly associating the class personnel target frame with the preset image division area; responding to the abnormal behavior region confirmation signal, acquiring the image data of the class video frame at the current moment, and performing face recognition processing on the portrait data in the class personnel target frame in the preset image division region corresponding to the abnormal behavior region confirmation signal to obtain a corresponding face recognition result; determining terminal position coding information of abnormal behavior personnel according to the face recognition result; and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel. The invention can automatically remind the staff of the class who does not face the blackboard side in the classroom, thereby improving the teaching consistency.
Description
Technical Field
The invention relates to the field of intelligent teaching, in particular to a reminding method and a reminding device for teaching assistance.
Background
The existing teaching mode is as follows: the teaching method comprises the steps of allocating a classroom to students, and then allocating a subject teacher to students of each class, so that the teachers can conduct teaching activities in the classroom, meanwhile, in the teaching process, some teaching plans are important when being allocated with classes, and if a certain student of the class is not serious in the class, the student can not follow the following teaching plan. When the teacher judges that the student is not serious, whether the student is serious is often judged by whether the student faces the teacher (blackboard side), and when the teacher of the subject cannot see the face of the student, the student often stops in a classroom to remind the student who is not serious, so that the classroom teaching of the teacher of the subject is influenced.
Disclosure of Invention
The invention aims to provide a reminding method and a reminding device for teaching assistance, which are used for solving the problem that a teacher cannot automatically remind an unidentified student during teaching.
In order to achieve the above object, the present invention provides a reminding method for teaching assistance, the reminding method comprising: acquiring a class personnel model based on Yolov4 network training;
acquiring class video frame image data in a classroom in real time, and inputting the class video frame image data into the class personnel model to obtain a class personnel target frame in the class video frame image data; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
responding to the abnormal behavior region confirmation signal, acquiring the image data of the class video frame at the current moment, carrying out face recognition processing on the portrait data in the class personnel target frame in the image preset division region corresponding to the abnormal behavior region confirmation signal, and obtaining a corresponding face recognition result;
determining terminal position coding information of abnormal behavior personnel according to the face recognition result;
and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel.
Optionally, the obtaining a class personnel model based on YOLOv4 network training includes:
1) establishing a class-taking video data set in a classroom;
2) establishing a YOLOv4 network structure;
3) training the YOLOv4 network structure obtained in the step 2) by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the YOLOv4 network structure;
4) and (4) performing iterative training on the pre-training model by using the lesson video data set until the loss function is converged to obtain a class personnel model based on the Yolov4 network training.
Optionally, the abnormal behavior region confirmation signal is generated based on the following conditions:
acquiring position information of each class personnel target frame in each image preset divided area;
calculating the variance of the distance between the positions of the adjacent class personnel target frames in each image preset division area;
when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and obtaining an abnormal behavior area signal of a teacher terminal;
determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined;
when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
Optionally, the face recognition processing is performed on the portrait data in the class personnel target frame in the preset image division area corresponding to the abnormal behavior area confirmation signal, and a corresponding face recognition result is obtained; the method comprises the following steps:
acquiring images in a class personnel target frame; applying an LBPH face recognition algorithm to the images in the class personnel target frame to obtain an LBP coding histogram set; comparing the set of LBP coding histograms to an LBPH map for the class personnel;
if the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal;
and if the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
Optionally, the determining, according to the face recognition result, position coding information of the terminal of the abnormal behavior person specifically includes:
if the face recognition result is that the face recognition of the class personnel target frame is abnormal, determining the position information of the class personnel target frame corresponding to the abnormal face recognition of the class personnel target frame as the recognition result;
and if the position of the class personnel target frame belongs to the range of the personnel terminal position code information, determining the personnel terminal position code information as the abnormal behavior personnel terminal position code information.
The invention also provides a reminding device for teaching assistance, which comprises:
a person identification module to: acquiring a class personnel model based on YOLOv4 network training, acquiring class video images in a classroom in real time, and inputting each class video frame image data into the class personnel model to obtain a class personnel target frame in each class video frame image data; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
an abnormal area judgment module: the system comprises a video frame image data acquisition unit, a face recognition unit, a data acquisition unit and a face recognition unit, wherein the video frame image data acquisition unit is used for acquiring the class video frame image data at the current moment in response to an abnormal behavior region confirmation signal, and carrying out face recognition processing on the portrait data in a class personnel target frame in an image preset division region corresponding to the abnormal behavior region confirmation signal to obtain a corresponding face recognition result;
and the reminding module is used for determining the terminal position coding information of the abnormal behavior personnel according to the face recognition result and sending corresponding reminding information according to the terminal position coding information of the abnormal behavior personnel.
Optionally, the person identification module is further configured to:
1) establishing a class-taking video data set in a classroom;
2) establishing a YOLOv4 network structure;
3) training the YOLOv4 network structure obtained in the step 2) by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the YOLOv4 network structure;
4) and (4) performing iterative training on the pre-training model by using the lesson video data set until the loss function is converged to obtain a class personnel model based on the Yolov4 network training.
Optionally, the abnormal region determining module is further configured to generate an abnormal behavior region confirmation signal, where the abnormal behavior region confirmation signal is generated based on the following conditions: acquiring position information of each class personnel target frame in each image preset divided area; calculating the variance of the distance between adjacent position information in each preset division area of each image; when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and obtaining an abnormal behavior area signal of a teacher terminal; determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined; when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
Optionally, the abnormal area determining module is further configured to obtain an image in the person target frame; applying an LBPH face recognition algorithm to the image in the personnel target frame to obtain an LBP coding histogram set; comparing the set of LBP coding histograms to an LBPH map for the class personnel;
when the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal; and when the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
Optionally, the reminding module is specifically configured to: if the face recognition result is that the face recognition of the class personnel target frame is abnormal, determining the position of the class personnel target frame corresponding to the abnormal face recognition of the class personnel target frame;
and if the determined position of the class personnel target frame belongs to the range of the personnel terminal position code information, determining the personnel terminal position code information as the abnormal behavior personnel terminal position code information.
According to the technical scheme, the class personnel model is used for acquiring the class video frame image data in the classroom in real time, and identifying the class personnel target frame in the class video frame image data; then responding to the abnormal behavior area confirmation signal, carrying out face recognition processing on portrait data in a class personnel target frame in the abnormal behavior area, and then determining terminal position coding information of the abnormal behavior personnel according to the face recognition result; and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel. Therefore, the classroom can be used for automatically reminding people who do not face the blackboard side in the classroom, so that the teaching continuity is improved.
Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the embodiments of the invention without limiting the embodiments of the invention. In the drawings:
fig. 1 is a schematic flow chart of a reminding method for teaching assistance according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating embodiments of the invention, are given by way of illustration and explanation only, not limitation.
As shown in fig. 1, a reminding method for teaching assistance, the reminding method includes: acquiring a class personnel model based on Yolov4 network training;
acquiring class video frame image data in a classroom in real time, and applying the class personnel model to each class video frame image data to obtain a class personnel target frame in each class video frame image data; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
responding to the abnormal behavior region confirmation signal, acquiring the image data of the class video frame at the current moment, carrying out face recognition processing on the portrait data in the class personnel target frame in the image preset division region corresponding to the abnormal behavior region confirmation signal, and obtaining a corresponding face recognition result;
determining terminal position coding information of abnormal behavior personnel according to the face recognition result;
and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel.
Optionally, the training and obtaining the class personnel model based on the YOLOv4 network includes:
1) establishing a class-taking video data set in a classroom; the method comprises the following steps that a camera is arranged on the blackboard side and used for a class video data set facing the blackboard side when students class;
2) establishing a Yolov4 network structure by using the existing method, such as based on CSPdaktnet 53 as a backbone network;
3) a large data set of ImageNet is used (ImageNet, just like a network, has a plurality of nodes. Each node corresponds to an item or a subcategory. According to the official network message, one node contains at least 500 pictures/images of corresponding objects for training. Actually, the image library for image/visual training) to train the Yolov4 network structure obtained in step 2) to obtain a pre-training model, and then setting specific training parameters for the Yolov4 network structure;
4) and (4) iteratively training the pre-training model by using the lesson video data set until the loss function is converged (if the box regression loss GIoU is converged), and obtaining a class personnel model based on the YOLOv4 network training. Because the moving range of the detection target of each person in the video data set in class is small, the training times can be reduced by adopting the Yolov 4-based network training, and the difficulty of the convergence of the loss function is reduced.
Optionally, the abnormal behavior region confirmation signal is generated based on the following conditions:
acquiring position information of each class personnel target frame in each image preset divided area; then calculating the variance of the distance between adjacent position information in each preset division area of each image;
when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and acquiring an abnormal behavior area signal of a teacher terminal; determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined; when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
Specifically, the preset image division regions may be divided according to daily experiences of a classroom, for example, in which region the person is likely not to face the teacher or the blackboard side, one region area is set, and the remaining image regions are randomly divided, and the divided image region area is smaller than or equal to the minimum image region area set by the teacher. The preset image dividing area and all class personnel target frames have one-to-one correspondence relationship. The teacher terminal can be a controller and keys, the keys are respectively connected to the IO ports of the controller, and then the trigger signal of each key represents an image preset division area. And the controller inquires the corresponding image preset division region according to the abnormal behavior region signal, if the region identifier of the preset division region of the image to be determined is the same as the region identifier of the preset division region of the image, the confirmation is determined to be consistent, and the abnormal behavior region confirmation signal is output. Through the confirmation signal of the teacher terminal and the automatic judgment information of the position information of each class personnel target frame in each image preset division area, and the confirmation, the misoperation of the prompt information when the key is mistakenly pressed is avoided, and the class listening rhythm of the class personnel is influenced. The variance corresponding to the preset division area of the image can be compared with a preset value when meeting the preset condition, and when the variance is determined to be larger than the preset value, the variance is determined to be changed due to the fact that the action of a certain class personnel target frame is too large or disappears.
Optionally, the face recognition processing is performed on the portrait data in the class personnel target frame in the preset image division area corresponding to the abnormal behavior area confirmation signal, and a corresponding face recognition result is obtained; the method comprises the following steps: acquiring an image in a personnel target frame; applying an LBPH (local Binary Pattern) face recognition algorithm to the image in the personnel target frame to obtain an LBP coding histogram set; comparing the set of LBP coding histograms to an LBPH map for the class personnel;
if the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal; and if the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
Specifically, the existing LBPH (Local Binary pattern) face recognition algorithm has the following steps:
the method comprises the following steps: and (5) carrying out image graying processing. The image in the personnel target frame is a color image, the color image is grayed firstly, each pixel point of the color image is represented by a gray value, and the gray value between two adjacent pixel points is solved by a bilinear difference value method.
Step two: and (6) dividing the region.
The whole image is divided into a plurality of grid areas, the recognition accuracy is improved along with the increase of the number of the areas, and the corresponding calculation amount is correspondingly improved.
Step three: local features were extracted using the LBPH method.
In each partition area, taking each pixel as a center, judging the size relation between the pixel and the gray value of the pixel points around the circle, coding 0 or 1 for the surrounding points, thereby obtaining the binary coding of the points, and deducing and distinguishing whether the area is a point, a line, an edge, an angular point and the like.
Because the gray value range of each pixel point is 0-256, 266 possible data are to be processed, and the dimension is reduced by an ULBP method according to the processing principle of reducing the data processing amount as much as possible, so that the calculation amount is reduced, the operation speed is increased, and the system can respond in real time.
Forming a binary coded histogram:
the whole image is divided into a plurality of grid areas, each pixel point in each area has LBP and ULBP values, an LBPH graph of the area is formed when the data are counted, and then histograms of the areas are connected to form a histogram of the feature vector of the whole image.
Constructing a binary coded histogram set
The histograms of the plurality of objects are stored in a database, forming a set of LBP coded histograms.
Step four: face recognition and matching.
The LBPH image of the class personnel is compared with the binary coding histogram set constructed in the front, so that whether the face appears in the class personnel target frame or not is obtained, namely whether the class personnel face the black board side or not is judged, and the teaching assistance convenience is improved.
Optionally, the determining, according to the face recognition result, position coding information of the terminal of the abnormal behavior person specifically includes:
if the face recognition result is that the face recognition of the class personnel target frame is abnormal, determining the position of the class personnel target frame corresponding to the abnormal face recognition of the class personnel target frame;
and if the determined position of the class personnel target frame belongs to the range of the personnel terminal position code information, determining the personnel terminal position code information as the abnormal behavior personnel terminal position code information. The range of the position code information of the personnel terminal indicates that the personnel has a certain moving range, namely the range of the position code information of the personnel terminal corresponds to the position information range of the target frame of the personnel in the class. The person terminal position code information may be address information of a reminder device installed on a seat of a class person.
The invention also provides a reminding device for teaching assistance, which comprises:
a personnel identification module: acquiring a class personnel model based on Yolov4 network training; acquiring class video frame image data in the classroom in real time, and applying the class personnel model to each class video frame image data to obtain a class personnel target frame in each class video frame image data; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
an abnormal area judgment module: responding to the abnormal behavior region confirmation signal, acquiring the image data of the class video frame at the current moment, and performing face recognition processing on the portrait data in the class personnel target frame in the image preset division region corresponding to the abnormal behavior region confirmation signal to obtain a corresponding face recognition result;
a reminding module: and determining terminal position coding information of the abnormal behavior personnel according to the face recognition result, and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel.
Preferably, supplementary reminding device of teaching is preferred to be set up in the controller, has improved the application efficiency of controller. In order to better prompt people in classes who are not serious, an LED lamp or a buzzer can be arranged on a seat of each class of people and respectively connected with the controller; the prompt message can be the flashing frequency of the LED lamp or the ringing current of the buzzer. Preferably, a feedback key is arranged on a seat of each class worker and connected with the controller, and when the class workers trigger the feedback keys, a reminding module in the controller stops sending corresponding prompt information within preset time; the preset time is set for timely feeding back trigger information of the class personnel, and the use experience of the class personnel without any person is improved.
Optionally, the obtaining a class personnel model based on YOLOv4 network training includes:
1) establishing a class-taking video data set in a classroom;
2) establishing a YOLOv4 network structure;
3) training the YOLOv4 network structure obtained in the step 2) by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the YOLOv4 network structure;
4) and (4) carrying out iterative training on the pre-training model by using a training set until the loss function is converged, so as to obtain a class personnel model based on the Yolov4 network training.
Optionally, the abnormal region determining module is further configured to generate an abnormal behavior region confirmation signal, where the abnormal behavior region confirmation signal is generated based on the following conditions:
acquiring position information of each class personnel target frame in each image preset divided area; calculating the variance of the distance between adjacent position information in each preset division area of each image; when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and obtaining an abnormal behavior area signal of a teacher terminal; determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined; when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
Optionally, the abnormal area determining module is further configured to obtain an image in the person target frame; applying an LBPH (local Binary Pattern) face recognition algorithm to the image in the personnel target frame to obtain an LBP coding histogram set; comparing the set of LBP coding histograms to an LBPH map for the class personnel;
when the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal; and when the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
Optionally, the reminding module is specifically configured to: and when the identification result is that the face identification of the class personnel target frame is abnormal, determining that the position information of the class personnel target frame corresponding to the abnormal face identification of the class personnel target frame belongs to the range of the position coding information of the personnel terminal, and determining that the position coding information of the personnel terminal is the position coding information of the abnormal behavior personnel terminal.
It should be noted that the specific details and benefits of the reminding device for teaching assistance provided by the present invention are similar to those of the reminding method for teaching assistance provided by the present invention, and are not described herein again.
Although the embodiments of the present invention have been described in detail with reference to the accompanying drawings, the embodiments of the present invention are not limited to the details of the above embodiments, and various simple modifications can be made to the technical solutions of the embodiments of the present invention within the technical idea of the embodiments of the present invention, and the simple modifications all belong to the protection scope of the embodiments of the present invention.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, the embodiments of the present invention do not describe every possible combination.
Those skilled in the art will understand that all or part of the steps in the method according to the above embodiments may be implemented by a program, which is stored in a storage medium and includes several instructions to enable a single chip, a chip, or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In addition, any combination of various different implementation manners of the embodiments of the present invention is also possible, and the embodiments of the present invention should be considered as disclosed in the embodiments of the present invention as long as the combination does not depart from the spirit of the embodiments of the present invention.
Claims (10)
1. A method for prompting for teaching assistance, the method comprising:
acquiring a class personnel model based on Yolov4 network training;
acquiring class video images in a classroom in real time, and inputting the image data of each class video frame into the class personnel model to obtain a class personnel target frame in the image data of each class video frame; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
responding to the abnormal behavior region confirmation signal, acquiring the image data of the class video frame at the current moment, and performing face recognition processing on the portrait data in the class personnel target frame in the image preset division region corresponding to the abnormal behavior region confirmation signal to obtain a corresponding face recognition result;
determining terminal position coding information of abnormal behavior personnel according to the face recognition result;
and sending corresponding prompt information according to the terminal position coding information of the abnormal behavior personnel.
2. The reminding method of claim 1, wherein the obtaining a class personnel model based on YOLOv4 network training comprises:
1) establishing a class-taking video data set in a classroom;
2) establishing a YOLOv4 network structure;
3) training the YOLOv4 network structure obtained in the step 2) by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the YOLOv4 network structure;
4) and performing iterative training on the pre-training model by using the lesson video data set until a loss function is converged to obtain a class personnel model based on the YOLOv4 network training.
3. The reminding method according to claim 1, wherein the abnormal behavior region confirmation signal is generated based on the following conditions:
acquiring the position of a target frame of each class person in each preset divided area of each image;
calculating the variance of the distance between the positions of the adjacent class personnel target frames in each image preset division area;
when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and acquiring an abnormal behavior area signal of a teacher terminal;
determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined;
when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
4. The reminding method according to claim 1, wherein the face recognition processing of the face data in the class personnel target frame in the preset image division area corresponding to the abnormal behavior area confirmation signal to obtain a corresponding face recognition result comprises:
acquiring images in a class personnel target frame;
applying an LBPH face recognition algorithm to the image in the class personnel target frame to obtain an LBP coding histogram set, and comparing the LBP coding histogram set with an LBPH image of the class personnel;
if the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal;
and if the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
5. The reminding method according to claim 4, wherein the determining of the position coding information of the terminal of the abnormal behavior person according to the face recognition result comprises:
if the face recognition result is that the face recognition of the class personnel target frame is abnormal, determining the position of the class personnel target frame corresponding to the abnormal face recognition of the class personnel target frame;
and if the determined position of the class personnel target frame belongs to the range of the personnel terminal position code information, determining the personnel terminal position code information as the abnormal behavior personnel terminal position code information.
6. A reminder device for educational assistance, the reminder device comprising:
a person identification module to:
acquiring a class personnel model based on YOLOv4 network training, acquiring class video images in a classroom in real time, and inputting each class video frame image data into the class personnel model to obtain a class personnel target frame in each class video frame image data; correspondingly associating the class personnel target frame in the image data of each class video frame with a preset image division area;
the abnormal region judgment module is used for responding to the abnormal behavior region confirmation signal, acquiring the image data of the video frame of the class at the current moment, and carrying out face recognition processing on the image data in the class personnel target frame in the preset division region of the image corresponding to the abnormal behavior region confirmation signal to obtain a corresponding face recognition result;
and the reminding module is used for determining the terminal position coding information of the abnormal behavior personnel according to the face recognition result and sending corresponding reminding information according to the terminal position coding information of the abnormal behavior personnel.
7. The reminding device as claimed in claim 6, wherein the obtaining of the class personnel model based on YOLOv4 network training comprises:
1) establishing a class-taking video data set in a classroom;
2) establishing a YOLOv4 network structure;
3) training the YOLOv4 network structure obtained in the step 2) by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the YOLOv4 network structure;
4) and performing iterative training on the pre-training model by using the lesson video data set until a loss function is converged to obtain a class personnel model based on the YOLOv4 network training.
8. The reminding device as claimed in claim 6, wherein the abnormal region judging module is further configured to generate an abnormal behavior region confirmation signal, and the abnormal behavior region confirmation signal is generated based on the following conditions:
acquiring the position of a target frame of each class person in each preset divided area of each image;
calculating the variance of the distance between the positions of the adjacent class personnel target frames in each image preset division area;
when the variance corresponding to the preset image division area meets a preset condition, obtaining the preset image division area to be determined, and acquiring an abnormal behavior area signal of a teacher terminal;
determining an image preset division region according to the abnormal behavior region signal of the teacher terminal, and confirming the region consistency with the image preset division region to be determined;
when the agreement is confirmed, an abnormal behavior region confirmation signal is output.
9. The reminding device according to claim 6, wherein the face recognition processing of the face data in the class personnel target frame in the preset image partition area corresponding to the abnormal behavior area confirmation signal to obtain the corresponding face recognition result comprises:
acquiring images in a class personnel target frame;
applying an LBPH face recognition algorithm to the image in the class personnel target frame to obtain an LBP coding histogram set, and comparing the LBP coding histogram set with an LBPH image of the class personnel;
if the comparison result is consistent, the face recognition result is that the face recognition of the class personnel target frame is normal;
and if the comparison result is inconsistent, the face recognition result is that the face recognition of the class personnel target frame is abnormal.
10. The reminding device as claimed in claim 9, wherein the determining of the position coding information of the terminal of the abnormal behavior person according to the face recognition result comprises:
if the face recognition result is that the face recognition of the class personnel target frame is abnormal, determining the position of the class personnel target frame corresponding to the abnormal face recognition of the class personnel target frame;
and if the determined position of the class personnel target frame belongs to the range of the personnel terminal position code information, determining the personnel terminal position code information as the abnormal behavior personnel terminal position code information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110884553.8A CN113705360A (en) | 2021-08-03 | 2021-08-03 | Reminding method and reminding device for teaching assistance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110884553.8A CN113705360A (en) | 2021-08-03 | 2021-08-03 | Reminding method and reminding device for teaching assistance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113705360A true CN113705360A (en) | 2021-11-26 |
Family
ID=78651423
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110884553.8A Withdrawn CN113705360A (en) | 2021-08-03 | 2021-08-03 | Reminding method and reminding device for teaching assistance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113705360A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114170588A (en) * | 2021-12-13 | 2022-03-11 | 西南交通大学 | Railway dispatcher bad state identification method based on eye features |
-
2021
- 2021-08-03 CN CN202110884553.8A patent/CN113705360A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114170588A (en) * | 2021-12-13 | 2022-03-11 | 西南交通大学 | Railway dispatcher bad state identification method based on eye features |
| CN114170588B (en) * | 2021-12-13 | 2023-09-12 | 西南交通大学 | Eye feature-based bad state identification method for railway dispatcher |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111507283B (en) | Student behavior identification method and system based on classroom scene | |
| CN110024015B (en) | Interactive video teaching method and system | |
| CN106485191A (en) | A kind of method for detecting fatigue state of driver and system | |
| CN110827432B (en) | Class attendance checking method and system based on face recognition | |
| CN113705360A (en) | Reminding method and reminding device for teaching assistance | |
| CN112949546A (en) | Off-duty detection method and device based on artificial intelligence perception of station service state | |
| CN111177398B (en) | Method and device for updating organization architecture | |
| CN116580350A (en) | Laboratory safety monitoring and early warning method and system | |
| CN109740527B (en) | Image processing method in video frame | |
| CN110390316A (en) | Data processing system and device based on recognition of face | |
| CN114187640A (en) | Learning situation observation method, system, equipment and medium based on online classroom | |
| CN113095259A (en) | Remote online course teaching management method | |
| CN113705510A (en) | Target identification tracking method, device, equipment and storage medium | |
| CN110708392B (en) | Student management system and method based on cloud platform | |
| CN111800428A (en) | Real-time statistical method and system for digital conference participation | |
| CN116563604A (en) | End-to-end target detection model training, image target detection method and related equipment | |
| CN113610688B (en) | Based on big food for data analysis Security supervision method and storage medium | |
| CN115797292A (en) | Image quality detection method, surface defect detection method, device and storage medium | |
| CN113052049A (en) | Off-duty detection method and device based on artificial intelligence tool identification | |
| CN113343850B (en) | Method, device, equipment and storage medium for checking video character information | |
| CN114677751B (en) | Learning state monitoring method, monitoring device and storage medium | |
| CN117218756B (en) | Intelligent safety pick-up system and method based on face recognition | |
| CN114331199A (en) | Campus security management system based on internet | |
| CN110659356B (en) | Information matching method, device and storage medium | |
| CN116246336A (en) | Classroom student behavior analysis system based on edge calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20211126 |