WO2021047185A1 - 基于人脸识别的监测方法、装置、存储介质及计算机设备 - Google Patents

基于人脸识别的监测方法、装置、存储介质及计算机设备 Download PDF

Info

Publication number
WO2021047185A1
WO2021047185A1 PCT/CN2020/087555 CN2020087555W WO2021047185A1 WO 2021047185 A1 WO2021047185 A1 WO 2021047185A1 CN 2020087555 W CN2020087555 W CN 2020087555W WO 2021047185 A1 WO2021047185 A1 WO 2021047185A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
face
images
abnormal
facial
Prior art date
Application number
PCT/CN2020/087555
Other languages
English (en)
French (fr)
Inventor
陈梦莎
Original Assignee
深圳壹账通智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳壹账通智能科技有限公司 filed Critical 深圳壹账通智能科技有限公司
Publication of WO2021047185A1 publication Critical patent/WO2021047185A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition

Definitions

  • This application relates to the field of teaching and education, and in particular to a monitoring method, device, storage medium and computer equipment based on face recognition.
  • classroom teaching is the whole process of teachers imparting knowledge and skills to students, and it is the most basic and most important link in the process of talent training for the transfer of theoretical knowledge. It is a method commonly used in education and teaching.
  • the quality of classroom teaching has a direct impact on the quality of talent training.
  • teachers need to understand the actual learning situation of students in the classroom.
  • students are prone to have weak self-control ability and tend to lose their minds. If the teacher is unable to detect and stop it in time, students will be prone to fail to keep up with the content of the class and decline in grades in the long run.
  • this application provides a monitoring method, device, storage medium and computer equipment based on face recognition.
  • a monitoring method based on face recognition includes: acquiring an image to be recognized, extracting all face images in the image to be recognized, and determining the location of each face image. Corresponding identity identification; determine the facial expression information in the facial image, and determine whether the facial expression information is abnormal; when the facial expression information is abnormal, the identity corresponding to the facial image with abnormal facial expression information The identifier is used as an abnormal identity identifier, and an abnormal behavior record of the abnormal identity identifier is generated.
  • a monitoring device based on face recognition including: an acquisition module for acquiring an image to be recognized, extracting all face images in the image to be recognized, and determining each The identity identifier corresponding to the facial image; the facial expression recognition module is used to determine the facial expression information in the facial image and determine whether the facial expression information is abnormal; the abnormality recording module is used to, when the facial facial information is abnormal, The identification corresponding to the face image with the abnormal facial expression information is used as the abnormal identification, and an abnormal behavior record of the abnormal identification is generated.
  • a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, a method for monitoring based on face recognition is implemented ,
  • the monitoring method based on face recognition includes the following steps: acquiring an image to be recognized, extracting all face images in the image to be recognized, and determining the identity identifier corresponding to each of the face images; Determine the expression information in the facial image, and determine whether the facial expression information is abnormal; when the facial expression information is abnormal, use the identity identifier corresponding to the facial image with the abnormal facial expression information as the abnormal identity identifier , And generate the abnormal behavior record of the abnormal identity.
  • a computer device which includes: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory And is configured to be executed by the one or more processors, and the one or more computer programs are configured to execute a method for monitoring based on face recognition, wherein the method for monitoring based on face recognition It includes the following steps: acquiring the image to be recognized, extracting all face images in the image to be recognized, determining the identity identifier corresponding to each of the face images; determining the expression information in the face image, and determining Whether the facial expression information is abnormal; when the facial expression information is abnormal, the identity corresponding to the facial image with the abnormal facial expression information is used as the abnormal identity, and an abnormal behavior record of the abnormal identity is generated.
  • the embodiment of the application provides a monitoring method, device, storage medium, and computer equipment based on face recognition, which extracts the face image of the student in the image to be recognized, and also determines the identity of the student corresponding to each face image. And based on facial expression recognition to determine whether the student's facial expression is abnormal; when the facial expression is abnormal, the abnormal student can be accurately located, and abnormal behavior records can be generated for the corresponding student.
  • This method can automatically identify the changes in the expressions and emotions of the entire class of students, and capture abnormal expressions, without the teacher's excessive involvement; based on abnormal behavior records, it can analyze the corresponding students' teaching profile in a targeted manner, and it is also convenient for teachers to targeted Teaching can improve the teaching effect of teachers.
  • the cyclic neural network Based on the cyclic neural network, it can generate image sequences containing all human face images and all adjacency relationships, which can ensure the accuracy of facial expression recognition; and convert multi-dimensional face images into one-dimensional face feature sequences, which can be faster Train the facial expression recognition model, and the recognition result of the model is more accurate.
  • FIG. 1 is a schematic flowchart of a monitoring method based on face recognition provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of a specific process of determining facial expression information in a face image in a monitoring method based on face recognition provided by an embodiment of the application;
  • FIG. 3 is a schematic diagram of a division of a face image provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of a structure of a cyclic neural network provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of a monitoring device based on face recognition provided by an embodiment of the application.
  • Fig. 6 is a schematic structural diagram of a computer device for implementing a classroom monitoring method provided by an embodiment of the application.
  • the embodiment of the application provides a monitoring method based on face recognition, which is applicable to the field of smart city technology, as shown in FIG. 1, and includes:
  • Step 101 Obtain an image to be recognized, extract all face images in the image to be recognized, and determine the identity identifier corresponding to each face image.
  • an image containing a human face can be collected based on a pre-set camera device, and then a human face image in the image to be recognized can be extracted.
  • the teacher can set up a shooting device, such as a camera, a camera, etc., in the classroom when teaching a lesson to the student, and the shooting device can collect a classroom image containing a student's face image, that is, the above-mentioned image to be recognized.
  • the camera can be set up in front of the class with the lens facing the students attending the class. After the image to be recognized is acquired, all facial images in the image to be recognized can be extracted based on the face recognition technology.
  • the image to be recognized can be directly collected by the shooting device; or, the class video is collected by the shooting device, and the image to be recognized is extracted from the class video.
  • different students in the classroom have different facial features, that is, each facial image can correspond to the corresponding student, so that you can determine which student each facial image corresponds to, and you can determine the identity of the student corresponding to the facial image .
  • determining the identity identifier corresponding to each face image includes:
  • Step A1 Perform matching processing on the face image and the face database, and use the identity of the face matching the face image in the face database as the identity of the face image.
  • a face database containing student faces is established in advance, and after the face image in the image to be recognized is extracted, the face image can be matched with the face database, that is, the face database can be queried If there is a student face that matches the face image, it means that the collected face image matches a face in the face database, because each face in the face database is preset That is, which student corresponds to each face is preset, so at this time, the identity of the face that matches the face image in the face database can be directly used as the identity of the face image.
  • Step A2 Presetting the correspondence between each image area in the image to be recognized and the identity tag, and according to the facial image in the image to be recognized The image area that belongs to and the corresponding relationship between the image area and the identity mark determine the identity mark corresponding to the face image.
  • the position of the photographing device that captures the image to be recognized is fixed in advance.
  • the facial images of different students in the class are located in different positions of the image to be recognized.
  • the corresponding relationship between the image area and the identity mark can be determined when the image to be recognized is collected, the student corresponding to each face image, that is, the identity mark corresponding to the face image. For example, a certain area in the upper left corner of the image to be recognized corresponds to the face image of student A, and if a certain face image a exists in the area in the upper left corner of the image to be recognized, the face image a corresponds to the identity of student A.
  • Step A3 Generate an image to be recognized with depth information based on images collected by multiple shooting devices located at different positions, and extract the image to be recognized Recognize the face image with depth information in the image; determine the three-dimensional coordinates of the face image in the world coordinate system according to the depth information of the face image and the position of the face image in the image to be recognized, and the user’s corresponding to the three-dimensional coordinates
  • the identity mark serves as the identity mark of the face image.
  • shooting devices based on multiple different positions can collect depth information, that is, three-dimensional images to be recognized can be collected and generated, and then based on the inherent physical parameters of the shooting device itself (such as location, focal length, etc.)
  • the face image in the image to be recognized is converted to the world coordinate system, so that the three-dimensional coordinates of the face image in the world coordinate system can be determined, and then the student corresponding to the three-dimensional coordinate can be determined to determine the identity of the face image .
  • student A is in the third row and fourth column of the classroom, and the third row and fourth column correspond to coordinates (30, 40) in the real world coordinate system; if the three-dimensional face image a in the image to be recognized
  • the coordinates are (30, 40, 10), where 10 is the height value, indicating that the face image a is the face image of student A, that is, the face image a corresponds to the identity of student A.
  • the principle of the three-dimensional images collected by multiple shooting devices is basically similar to the working principle of the binocular camera, and will not be repeated here.
  • Step 102 Determine the expression information in the face image, and judge whether the expression information is abnormal.
  • the facial expression information in the facial image can be identified, and based on the facial expression information, it is determined whether the student is distracted.
  • the facial expression information in the face image can be recognized based on the traditional facial expression recognition method, so that the corresponding student's classroom emotional state can be judged.
  • a facial expression coding system Facial Action Coding System, FACS
  • the recognition of facial expressions is generally implemented based on multiple motion units AU (Action Unit), that is, the process of facial expression recognition is completed through multiple motion units AU.
  • Step 103 When the facial expression information is abnormal, use the identity identifier corresponding to the face image with the abnormal facial expression information as the abnormal identity identifier, and generate an abnormal behavior record of the abnormal identifier.
  • the facial expression information of a student indicates distraction, inattention, etc.
  • the facial expression information is considered abnormal.
  • it can be judged which students are distracted, and then the abnormal identity can be determined.
  • the abnormal behavior record of the abnormal identity can be generated.
  • the abnormal behavior record can record abnormal behaviors such as wandering.
  • the teacher can be reminded in real time which students are not paying attention, or a classroom report can be generated, so that the teacher can conduct targeted teaching based on the classroom report and improve teaching effect.
  • the embodiment of the application provides a monitoring method based on face recognition. While extracting the face image of the student in the image to be recognized, the identity of the student corresponding to each face image is also determined, and based on the facial expression recognition, it is determined whether the student’s expression is Abnormal: When the expression is abnormal, the abnormal student can be accurately located, and abnormal behavior records can be generated for the corresponding student. This method can automatically identify the changes in the expressions and emotions of the entire class of students, and capture abnormal expressions, without the teacher's excessive involvement; based on abnormal behavior records, it can analyze the corresponding students' teaching profile in a targeted manner, and it is also convenient for teachers to targeted Teaching can improve the teaching effect of teachers.
  • step 102 determine the expression information in the face image.
  • Step 1021 Divide the face image into multiple parts, and determine the face part image corresponding to each part.
  • the face image is divided into multiple parts, and each part corresponds to an image, that is, a face part image, such as a left eye image, a right eye image, a mouth image, etc.; images based on multiple parts
  • a face part image such as a left eye image, a right eye image, a mouth image, etc.
  • images based on multiple parts The facial expression or emotion corresponding to the face image can be determined in a refined manner.
  • Step 1022 Generate the face feature sequence of the face image according to all the face part images, use the face feature sequence as the input of the pre-trained expression recognition model, and determine the expression corresponding to the face image according to the expression recognition model information.
  • facial expression recognition models generally directly use face images as input. Although this method is simple, since the input of facial expression recognition models is a complete face image, a large number of training sets are required when training the facial expression recognition model, and the recognition accuracy is not high.
  • a face feature sequence is generated based on all face part images of a face image, and a multi-dimensional face image is converted into a one-dimensional face feature sequence, which can train the expression recognition model more accurately and quickly. The recognition result is more accurate.
  • the process of training the expression recognition model is also included, and the process specifically includes:
  • Step B1 Determine training samples and establish an expression recognition model.
  • the training samples include the face feature sequence of the face sample image and the expression information corresponding to the face sample image.
  • Step B2 Use the facial feature sequence of the face sample image as the input of the facial expression recognition model, and use the facial expression information corresponding to the facial sample image as the output of the facial expression recognition model, train the facial expression recognition model, and generate the trained facial expression Identify the model.
  • the expression recognition model may specifically be a classifier. After the training is completed, expression recognition can be performed based on the trained expression recognition model.
  • the face feature sequence is generated based on the adjacency relationship between the face part images.
  • Step 1022 “generate the face feature sequence of the face image based on all the face part images” specifically includes:
  • Step C1 Determine the adjacency relationship between each face part image and other face part images according to the position of the face part image in the face image.
  • the face image is divided into multiple face part images.
  • the face image is divided into multiple face part images.
  • this embodiment takes the face image divided into four face part images as an example. The division method can be seen in Figure 3.
  • the face image is divided into four face part images 1234, where , 3 is adjacent to 1, 2, 4, then 1 and 3 have an adjacent relationship, 2 and 3 have an adjacent relationship, 3 and 4 have an adjacent relationship; at the same time, 4 is adjacent to 1 and 2 respectively , Then 1 has an adjacent relationship with the division, and 2 has an adjacent relationship with 4.
  • the adjacency relationship only means that two face part images are adjacent, and does not limit the priority of the two face part images.
  • the adjacency relationship between 1 and 3 the adjacency relationship between 3 and 1, both Indicates the same meaning.
  • Step C2 The face part image with the most adjacency relations is taken as the first sequential image of the pending image sequence, and then the first sequential image is taken as the current sequential image, and the image that has the adjacency relationship with the current sequential image
  • the other face part images are used as the next sequential image, and the above process of selecting the next sequential image based on the current sequential image is repeated until all the face part images are traversed, and all the faces between the face part images are traversed
  • the adjacency relationship finally generates an image sequence containing all the face part images and all the adjacency relationships between the face part images.
  • facial expressions are all based on facial muscles, when a certain muscle moves, it is difficult to only affect a certain part of the face. Because adjacent parts are related, when a certain muscle moves When driving a certain part to move, it will also drive other parts adjacent to that part to move at the same time. For example, when the corners of a person's mouth are raised, it will affect the state of the cheeks, and even the person's eyes.
  • a sequence of multiple face part images is established to reflect the relationship between the parts, that is, an image sequence is generated based on the adjacency relationship of the face part images.
  • the image sequence contains "mouth image ⁇ nose image” or "nose image ⁇ mouth image”; because a certain part may be adjacent to multiple other parts Yes, in the final image sequence, a certain part may appear multiple times.
  • the face part image with less adjacency relations is taken as the first sequential image of the image sequence, in order to ensure other face part images with more adjacency relations All the adjacency relations of are included in the image sequence, which requires an additional increase in the number of images of human faces in the image sequence, resulting in an image sequence that is too long.
  • the face image with the most adjacency is used as the first sequential image of the image sequence, which can reduce the length of the image sequence and is beneficial to model training and facial expression recognition The process can effectively reduce the amount of training, and at the same time can accurately recognize the expression of the face image.
  • the face image 1 is adjacent to 3 and 4, then the face image 1 has two adjacency relations.
  • the face image 2, 3, 4 have two Adjacent relations, three adjacent relations, three adjacent relations.
  • the front of the image sequence can be 1 ⁇ 3 ⁇ 4 ⁇ 2 ⁇ 3, but at this time, since the adjacency relationship between 1 and 4 is not included, the last Add "1 ⁇ 4" or "4 ⁇ 1", that is, the complete image sequence can be 1 ⁇ 3 ⁇ 4 ⁇ 2 ⁇ 3 ⁇ 4 ⁇ 1, the image sequence contains seven images of human faces.
  • the length of the image sequence can be reduced.
  • the image sequence can be 3 ⁇ 1 ⁇ 4 ⁇ 2 ⁇ 3 ⁇ 4, or 3 ⁇ 4 ⁇ 1 ⁇ 3 ⁇ 2 ⁇ 4, at this time
  • the image sequence of contains six images of human face parts. This image sequence contains all the adjacency relations while also having a small length.
  • the way of generating the image sequence can be preset, and the image sequence can be generated from the image to be recognized. After extracting the face image in the, you can directly generate the image sequence of the face image in the way of generating the image sequence.
  • Step C3 Convert the image sequence into the face feature sequence of the face image based on the cyclic neural network.
  • the characteristics of adjacent sequences can be memorized by using the cyclic neural network, and the facial feature sequence of the face image is generated based on the cyclic neural network. Specifically, each face part image in the image sequence is sequentially used as the input of the cyclic neural network, so that a face feature sequence containing the adjacency relationship of the face part image can be generated.
  • a schematic diagram of the structure of the recurrent neural network is shown in Figure 4, where x 1 to x 4 represent the input of the recurrent neural network, E 1 to E 4 represent the output of the recurrent neural network, and s 1 to s 4 represent the output of the recurrent neural network.
  • an image sequence containing all facial part images and all adjacency relationships can be generated, which can ensure the accuracy of facial expression recognition; and convert multi-dimensional facial images into one-dimensional facial features
  • the sequence can train the facial expression recognition model more quickly, and the recognition result of the model is more accurate.
  • the dynamic characteristics of the student are determined based on multiple frames of continuous face images, so that it can be dynamically determined whether the expression information of the student is abnormal, for example, whether the student is distracted.
  • "using the facial feature sequence as the input of the pre-trained facial expression recognition model, and determining the facial expression information corresponding to the facial image according to the facial expression recognition model" may specifically include;
  • Step D1 Continuously acquire multiple frames of images to be recognized, sequentially extract multiple frames of face images with the same identity identifier from the multiple frames of images to be recognized, and determine the face feature sequence of the face image of each frame.
  • Step D2 Taking the facial feature sequence of the multi-frame facial image as the input of the pre-trained facial expression recognition model, and determining the facial expression information corresponding to the multi-frame facial image according to the facial expression recognition model.
  • the dynamic change of the student's expression can be extracted, so that the judgment result is more accurate.
  • the face feature sequences of multiple face images the face feature sequences of multiple face images can be spliced into a whole sequence; or, based on another cyclic neural network, each face feature sequence can be used as a loop in turn The input of the neural network, thereby generating a complete feature sequence containing the dynamic changes of the student.
  • the cyclic neural network may specifically adopt LSTM (Long Short-Term Memory, long short-term memory network).
  • step 103 “generate abnormal behavior record of abnormal identity”
  • the method further includes:
  • Step E1 Obtain the associated information of the abnormal identification recorded in the associated system, and generate a comprehensive teaching record of the abnormal identification based on the associated information and abnormal behavior records; or
  • Step E2 Send the abnormal behavior record to the associated system, and instruct the associated system to generate a comprehensive teaching record of the abnormal identifier based on the associated information of the abnormal identifier recorded in the associated system and the abnormal behavior record.
  • the related system may specifically be a teaching-related system, such as a grade system.
  • Corresponding related information can be obtained from the related system. For example, student performance information, etc.; combined with the student’s abnormal behavior records in the classroom, as well as grades and other information, based on big data analysis technology can generate a comprehensive report of the student, that is, comprehensive teaching records, based on the comprehensive teaching records can be more convenient for teachers to target The student conducts adaptive teaching.
  • the embodiment of the application provides a monitoring method based on face recognition. While extracting the face image of the student in the image to be recognized, the identity of the student corresponding to each face image is also determined, and based on the facial expression recognition, it is determined whether the student’s expression is Abnormal: When the expression is abnormal, the abnormal student can be accurately located, and abnormal behavior records can be generated for the corresponding student. This method can automatically identify the changes in the expressions and emotions of the entire class of students, and capture abnormal expressions, without the teacher's excessive involvement; based on abnormal behavior records, it can analyze the corresponding students' teaching profile in a targeted manner, and it is also convenient for teachers to targeted Teaching can improve the teaching effect of teachers.
  • the cyclic neural network Based on the cyclic neural network, it can generate image sequences containing all human face images and all adjacency relationships, which can ensure the accuracy of facial expression recognition; and convert multi-dimensional face images into one-dimensional face feature sequences, which can be faster Train the facial expression recognition model, and the recognition result of the model is more accurate.
  • the flow of the monitoring method based on face recognition is described in detail above, and the method can also be implemented by a corresponding device.
  • the structure and function of the device are described in detail below.
  • a monitoring device based on face recognition provided by an embodiment of the present application, as shown in FIG. 5, includes:
  • the obtaining module 51 is configured to obtain the image to be recognized, extract all facial images in the image to be recognized, and determine the identity identifier corresponding to each of the facial images;
  • the expression recognition module 52 is configured to determine the expression information in the face image, and determine whether the expression information is abnormal
  • the abnormality recording module 53 is configured to, when the facial expression information is abnormal, use the identity identifier corresponding to the face image with the abnormal facial expression information as the abnormal identity identifier, and generate an abnormal behavior record of the abnormal identifier.
  • the acquiring module 51 determining the identity identifier corresponding to each of the face images includes:
  • the corresponding relationship between each image area in the image to be recognized and the identity tag is preset, and the corresponding relationship between the image area and the identity tag is determined according to the image area to which the face image belongs in the image to be recognized
  • the identity identifier corresponding to the face image or
  • the expression recognition module 52 includes:
  • An image dividing unit for dividing the face image into multiple parts, and determining the face part image corresponding to each part
  • the facial expression recognition unit is used to generate the facial feature sequence of the facial image based on all the facial part images, and use the facial feature sequence as the input of the pre-trained facial expression recognition model, according to the The facial expression recognition model determines the facial expression information corresponding to the facial image.
  • the facial expression recognition unit generating the facial feature sequence of the facial image according to all the facial part images includes:
  • the face part image with the most adjacency relations is taken as the first sequential image of the pending image sequence, and then the first sequential image is taken as the current sequential image, and will have an adjacency relationship with the current sequential image
  • the other face part images as the next sequential image, and the above process of selecting the next sequential image based on the current sequential image is repeated until all the face part images are traversed, and the face part is traversed All the adjacency relations between the images, and finally generate an image sequence including all the face part images and all the adjacency relations between the face part images;
  • the image sequence is converted into the face feature sequence of the face image based on the recurrent neural network.
  • the facial expression recognition unit uses the facial feature sequence as an input of a pre-trained facial expression recognition model, and the facial expression information corresponding to the facial image is determined according to the facial expression recognition model to include;
  • the face feature sequence of the face images of multiple frames is used as the input of the pre-trained expression recognition model, and the expression information corresponding to the face images of the multiple frames is determined according to the expression recognition model.
  • the expression recognition module 52 further includes a training unit
  • the training unit is used to:
  • the training sample includes the facial feature sequence of the facial sample image and the facial expression information corresponding to the facial sample image; take the facial feature sequence of the facial sample image as The input of the expression recognition model and the expression information corresponding to the face sample image are used as the output of the expression recognition model, the expression recognition model is trained, and the trained expression recognition model is generated.
  • the device further includes an integrated processing module
  • the comprehensive processing module is used to:
  • the abnormal behavior record is sent to the associated system, and the associated system is instructed to generate the comprehensive teaching of the abnormal identification based on the associated information of the abnormal identification recorded in the associated system and the abnormal behavior record recording.
  • the embodiment of the application provides a monitoring device based on face recognition. While extracting the face image of the student in the image to be recognized, it also determines the identity of the student corresponding to each face image, and determines whether the student’s facial expression is based on facial expression recognition.
  • Abnormal When the expression is abnormal, the abnormal student can be accurately located, and abnormal behavior records can be generated for the corresponding student. This method can automatically identify the changes in the expressions and emotions of the entire class of students, and capture abnormal expressions, without the teacher's excessive involvement; based on abnormal behavior records, it can analyze the corresponding students' teaching profile in a targeted manner, and it is also convenient for teachers to targeted Teaching can improve the teaching effect of teachers.
  • the cyclic neural network Based on the cyclic neural network, it can generate image sequences containing all human face images and all adjacency relationships, which can ensure the accuracy of facial expression recognition; and convert multi-dimensional face images into one-dimensional face feature sequences, which can be faster Train the facial expression recognition model, and the recognition result of the model is more accurate.
  • the embodiment of the present application also provides a storage medium storing computer-readable instructions, the storage medium is a volatile storage medium or a non-volatile storage medium, and the computer-readable instruction is executed by one or more processors At this time, one or more processors are caused to perform the following steps: obtain the image to be recognized, extract all face images in the image to be recognized, determine the identity identifier corresponding to each of the face images; determine the person The facial expression information in the facial image, and determine whether the facial expression information is abnormal; when the facial expression information is abnormal, the identity identifier corresponding to the facial image with the abnormal facial expression information is used as the abnormal identity identifier, and all the facial expressions are generated.
  • the abnormal behavior record of the abnormal identification is a volatile storage medium or a non-volatile storage medium
  • the computer-readable instruction is executed by one or more processors
  • one or more processors are caused to perform the following steps: obtain the image to be recognized, extract all face images in the image to be recognized, determine the identity identifier corresponding to each of
  • Fig. 6 shows a structural block diagram of a computer device according to another embodiment of the present application.
  • the computer device 1100 may be a host server with computing capabilities, a personal computer PC, or a portable computer or terminal that can be carried.
  • the specific embodiments of the present application do not limit the specific implementation of the computer device.
  • the computer device 1100 includes at least one processor 1110, a communications interface 1120, a memory (memory array) 1130, and a bus 1140. Among them, the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other through the bus 1140.
  • the communication interface 1120 is used to communicate with network elements, where the network elements include, for example, a virtual machine management center, shared storage, and the like.
  • the processor 1110 is used to execute programs.
  • the processor 1110 may be a central processing unit CPU, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
  • ASIC application specific integrated circuit
  • the memory 1130 is used for executable instructions.
  • the memory 1130 may include a high-speed RAM memory, or may also include a non-volatile memory (non-volatile memory), for example, at least one disk memory.
  • the memory 1130 may also be a memory array.
  • the memory 1130 may also be divided into blocks, and the blocks may be combined into a virtual volume according to certain rules.
  • the instructions stored in the memory 1130 may be executed by the processor 1110, so that the processor 1110 can execute the method in any of the foregoing method embodiments.

Abstract

本申请提供了一种基于人脸识别的监测方法、装置、存储介质及计算机设备，其中，该方法包括：获取待识别图像，并提取待识别图像中的所有人脸图像，确定每个人脸图像所对应的身份标识；确定人脸图像中的表情信息，并判断表情信息是否异常；在表情信息异常时，将与具有异常的表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成异常身份标识的异常行为记录。本申请提供的一种基于人脸识别的监测方法、装置、存储介质及计算机设备，可以自动识别全班学生的表情情绪变化，并抓取异常的表情，不需要老师过多参与；基于异常行为记录可以有针对性地分析相应学生的教学概况，也方便教师有针对性地进行教学，可以提高教师的教学效果。

Description

基于人脸识别的监测方法、装置、存储介质及计算机设备
本申请要求于2019年9月12日提交中国专利局、申请号为201910865404.X，发明名称为“基于人脸识别的监测方法、装置、存储介质及计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及教学教育领域，特别涉及一种基于人脸识别的监测方法、装置、存储介质及计算机设备。
背景技术
课堂教学是教师给学生传授知识和技能的全过程，是人才培养过程中进行理论知识传授最基本、最重要的环节。其是教育教学中普遍使用的一种手段，课堂教学质量的优劣对人才培养的质量有着直接的影响。为了提高课堂教学质量，需要教师了解学生在课堂中的实际学习情况。在上课过程中，学生容易出现自制能力比较弱，容易走神等状况。若老师无法及时察觉并及时制止，长期下来学生容易出现跟不上课堂的授课内容、成绩下滑等情况。
发明人发现在目前的教师授课过程中，尤其大班授课，学生课堂听课状况实时掌握非常困难，而辅导机构的一对一辅导虽然可以有效避免该问题，但其收费昂贵，并不具有普遍适用性。
发明内容
为解决上述问题，本申请提供一种基于人脸识别的监测方法、装置、存储介质及计算机设备。
根据本申请的第一个方面，提供一种基于人脸识别的监测方法，包括：获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
根据本申请的第二个方面，提供一种基于人脸识别的监测装置，包括：获取模块，用于获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；表情识别模块，用于确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；异常记录模块，用于在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
根据本申请的第三个方面，提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现一种基于人脸识别的监测的方法，其中，所述基于人脸识别的监测的方法包括以下步骤： 获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
根据本申请的第四个方面，提供一种计算机设备，其包括：一个或多个处理器；存储器；一个或多个计算机程序，其中所述一个或多个计算机程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个计算机程序配置用于执行一种基于人脸识别的监测的方法，其中，所述基于人脸识别的监测的方法包括以下步骤：获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
本申请实施例提供的一种基于人脸识别的监测方法、装置、存储介质及计算机设备，在提取待识别图像中学生的人脸图像的同时，还确定每个人脸图像对应的学生的身份标识，并基于表情识别确定学生表情是否异常；在表情异常时可以准确定位异常的学生，且可以为相应的学生生成异常行为记录。该方式可以自动识别全班学生的表情情绪变化，并抓取异常的表情，不需要老师过多参与；基于异常行为记录可以有针对性地分析相应学生的教学概况，也方便教师有针对性地进行教学，可以提高教师的教学效果。基于循环神经网络可以生成包含所有人脸部位图像以及所有邻接关系的图像序列，可以保证表情识别的准确性；且将多维的人脸图像转换为一维的人脸特征序列，可以更加快速地训练表情识别模型，模型的识别结果更加准确。
附图说明
图1为本申请实施例提供的基于人脸识别的监测方法的流程示意图；
图2为本申请实施例提供的基于人脸识别的监测方法中，确定人脸图像中的表情信息的具体流程示意图；
图3为本申请实施例提供的人脸图像的一种划分示意图；
图4为本申请实施例提供的循环神经网络的一种结构示意图；
图5为本申请实施例提供的基于人脸识别的监测装置的结构示意图；
图6为本申请实施例提供的执行课堂监测方法的计算机设备的结构示意图。
具体实施方式
本申请实施例提供的一种基于人脸识别的监测方法，适用于智慧城市技术领域，参见图1所示，包括：
步骤101：获取待识别图像，并提取待识别图像中的所有人脸图像，确定每个人脸图像所对应的身份标识。
本申请实施例中，基于预先设置好的摄像设备可以采集到包含人脸的图像，即待识别图像，进而可以提取出该待识别图像中的人脸图像。本实施例中，教师在向学生授课的过程中，可以在课堂内架设拍摄装置，例如相机、摄像头等，且该拍摄装置可以采集到包含学生人脸图像的课堂图像，即上述待识别图像。具体的，该拍摄装置可以架设在课堂的前方，且镜头朝向听课的学生。在获取到待识别图像之后，即可基于人脸识别技术提取出待识别图像中所有的人脸图像。其中，可以通过拍摄装置直接采集到待识别图像；或者，通过拍摄装置采集课堂视频，并从课堂视频中提取出待识别图像。同时，课堂中不同学生具有不同的人脸特征，即每个人脸图像可以对应相应的学生，从而可以确定每个人脸图像所对应是哪个学生，即可以确定人脸图像所对应的学生的身份标识。
可选的，可以采用多种方式来确定人脸图像对应的身份标识。具体的，上述步骤101中“确定每个人脸图像所对应的身份标识”包括：
步骤A1：对人脸图像与人脸数据库进行匹配处理，将人脸数据库中与人脸图像相匹配的人脸的身份标识作为人脸图像的身份标识。
本实施例中，预先建立包含学生人脸的人脸数据库，在提取出待识别图像中的人脸图像后，即可对该人脸图像与人脸数据库进行匹配处理，即查询在人脸数据库中是否存在与该人脸图像相匹配的学生人脸，若存在，则说明采集到的人脸图像与人脸数据库中的一个人脸是相匹配，由于预先设置了人脸数据库中每个人脸的身份标识，即预先设置了每个人脸对应哪个学生，故此时可以直接将人脸数据库中与人脸图像相匹配的人脸的身份标识作为人脸图像的身份标识。
或者，上述步骤101中“确定每个人脸图像所对应的身份标识”包括：步骤A2：预先设置待识别图像中每个图像区域与身份标识之间的对应关系，根据人脸图像在待识别图像中所属的图像区域以及图像区域与身份标识之间的对应关系确定人脸图像所对应的身份标识。
本申请实施例中，预先固定采集待识别图像的拍摄装置的位置，此时，情况下，课堂上不同学生的人脸图像位于该待识别图像的不同位置，通过预先设置待识别图像中每个图像区域与身份标识之间的对应关系，在采集到待识别图像时即可确定每个人脸图像所对应的学生，即人脸图像所对应的身份标识。例如，待识别图像左上角的某片区域对应学生A的人脸图像，若待识别图像在该左上角的区域内存在某个人脸图像a，则该人脸图像a对应学生A的身份标识。
或者，上述步骤101中“确定每个人脸图像所对应的身份标识”包括：步骤A3：基于设在不同位置的多个拍摄装置所采集的图像生成具有景深信息的待识别图像，并提取出待识别图像中具有景深信息的人脸图像；根据人脸图像的景深信息以及人脸图像在待识别图像中的位置确定人脸图像在世界坐标系下的三维坐标，将三维坐标所对应的用户的身份标识作为人脸图像的身份标识。
本实施例中，基于多个不同位置的拍摄装置可以采集到景深信息，即可以采集并生成三维的待识别图像，之后基于拍摄装置本身固有的物理参数(例如所在位置、焦距等)即可将待识别图像中的人脸图像转换至世界坐标系中，从 而可以确定人脸图像在世界坐标系下的三维坐标，之后即可确定该三维坐标对应哪个学生，即可以确定人脸图像的身份标识。例如，学生A在教室的第三排第四列，且第三排第四列在真实世界的世界坐标系下对应坐标(30,40)；若待识别图像中的某个人脸图像a的三维坐标为(30,40,10)，其中的10为高度值，则说明该人脸图像a是学生A的人脸图像，即人脸图像a对应学生A的身份标识。其中，多个拍摄装置采集到三维图像的原理与双目相机的工作原理基本类似，此处不做赘述。
步骤102：确定人脸图像中的表情信息，并判断表情信息是否异常。
本申请实施例中，在确定人脸图像后，可以识别出人脸图像中的表情信息，基于该表情信息来判断学生是否走神。具体的，可以基于传统的表情识别方法来识别出人脸图像中的表情信息，从而可以判断相应的学生的课堂情绪状态。例如，在面部表情编码系统(FacialAction Coding System，FACS)中，一般识别表情时是基于多个运动单元AU(Action Unit)实现的，即通过多个运动单元AU完成表情识别的过程。
步骤103：在表情信息异常时，将与具有异常的表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成异常身份标识的异常行为记录。
本申请实施例中，若学生的表情信息表示走神、注意力不集中等，则认为该表情信息异常，此时基于哪些表情信息异常即可判断哪些学生走神，即可以确定异常身份标识，之后即可生成该异常身份标识的异常行为记录。其中，该异常行为记录可以记载走神等异常行为，基于该异常行为记录可以实时提醒教师哪些学生注意力不集中，或者生成课堂报告，使得教师基于该课堂报告可以有针对性地进行教学，提高教学效果。
本申请实施例提供的一种基于人脸识别的监测方法，在提取待识别图像中学生的人脸图像的同时，还确定每个人脸图像对应的学生的身份标识，并基于表情识别确定学生表情是否异常；在表情异常时可以准确定位异常的学生，且可以为相应的学生生成异常行为记录。该方式可以自动识别全班学生的表情情绪变化，并抓取异常的表情，不需要老师过多参与；基于异常行为记录可以有针对性地分析相应学生的教学概况，也方便教师有针对性地进行教学，可以提高教师的教学效果。
在上述实施例的基础上，参见图2所示，步骤102“确定人脸图像中的表情信息”具体包括：
步骤1021：将人脸图像划分为多个部位，并确定每个部位所对应的人脸部位图像。
本申请实施例中，将人脸图像分为多个部位，每个部位对应一个图像，即人脸部位图像，比如左眼图像、右眼图像、嘴部图像等；基于多个部位的图像可以精细化确定该人脸图像对应的表情或情绪。
步骤1022：根据所有的人脸部位图像生成人脸图像的人脸特征序列，并将人脸特征序列作为预先训练好的表情识别模型的输入，根据表情识别模型确定人脸图像所对应的表情信息。
传统表情识别模型一般直接将人脸图像作为输入，该方式虽然简单，但是由于表情识别模型输入的是完整的人脸图像，在训练表情识别模型时需要大量的训练集，且识别精度不高。本申请实施例中，基于人脸图像的所有人脸部位图像生成人脸特征序列，将多维的人脸图像转换为一维的人脸特征序列，可以更加准确快速地训练表情识别模型，模型的识别结果更加准确。
其中，在将人脸特征序列作为预先训练好的表情识别模型的输入之前，还包括训练表情识别模型的过程，该过程具体包括：
步骤B1：确定训练样本，并建立表情识别模型，训练样本包括人脸样本图像的人脸特征序列以及与人脸样本图像相对应表情信息。
步骤B2：将人脸样本图像的人脸特征序列作为表情识别模型的输入、将与人脸样本图像相对应表情信息作为表情识别模型的输出，对表情识别模型进行训练，并生成训练好的表情识别模型。
本申请实施例中，该表情识别模型具体可以为分类器，在训练完后，即可基于训练好的表情识别模型进行表情识别。
在上述实施例的基础上，基于人脸部位图像之间的邻接关系来生成人脸特征序列，步骤1022“根据所有的人脸部位图像生成人脸图像的人脸特征序列”具体包括：
步骤C1：根据人脸部位图像在人脸图像中的位置分别确定每个人脸部位图像与其他人脸部位图像之间的邻接关系。
本申请实施例中，人脸图像被分为多个人脸部位图像，此时基于人脸部位图像在该人脸图像中的位置，即可确定哪些人脸部位图像是相邻的，即可以确定某个人脸部位图像与其他哪个人脸部位图像之间具有邻接关系，比如嘴部图像和鼻部图像具有邻接关系。为方便描述，本实施例以将人脸图像分为四个人脸部位图像为例说明，划分方式可参见图3，如图所示，人脸图像分为四个人脸部位图像①②③④，其中，③分别与①、②、④相邻，则①与③之间具有邻接关系，②与③之间具有邻接关系，③与④之间具有邻接关系；同时，④分别与①、②相邻，则①与司之间具有邻接关系，②与④之间具有邻接关系。其中，邻接关系只是表示两个人脸部位图像相邻，并不限定两个人脸部位图像的主次，例如，①与③之间的邻接关系，③与①之间的邻接关系，二者表示相同的含义。
步骤C2：将具有最多邻接关系的人脸部位图像作为待定的图像序列的第一顺位图像，之后将第一顺位图像作为当前顺位图像，并将与当前顺位图像具有邻接关系的其他人脸部位图像作为下一顺位图像，重复上述基于当前顺位图像选取下一顺位图像的过程，直至遍历所有的人脸部位图像、且遍历人脸部位图像之间的所有邻接关系，最终生成包含所有的人脸部位图像、以及人脸部位图像之间的所有邻接关系的图像序列。
由于人脸的表情都是基于面部肌肉做出来的，当某一肌肉动作时，很难只是影响人脸的某个部位，由于相邻的部位之间都是有关联关系的，当某个肌肉带动某一部位动作时，同时也会带动与该部位相邻的其他部位动作。例如，人嘴角上扬时会影响脸颊的状态，甚至影响到人的眼睛。本申请实施例中通过建 立多个人脸部位图像的序列来体现部位之间的关系，即基于人脸部位图像的邻接关系来生成图像序列。例如，嘴部图像与鼻子图像是相邻的，则该图像序列中包含“嘴部图像→鼻子图像”或者“鼻子图像→嘴部图像”；由于某个部位可能与多个其他部位是相邻的，最终生成的图像序列中，某个部位可能出现多次。
具体的，由于图像序列需要包含所有的邻接关系，若将邻接关系较少的人脸部位图像作为该图像序列的第一顺位图像，为了保证其他具有较多邻接关系的人脸部位图像的所有邻接关系均包含在该图像序列中，需要额外增加图像序列中人脸部位图像的数量，导致图像序列过长。本申请实施例中，为了尽量减少图像序列的长度，将具有最多邻接关系的人脸部位图像作为图像序列的第一顺位图像，可以减少图像序列的长度，且有利于模型训练以及表情识别的过程，可以有效减少训练量，同时也能精确识别出人脸图像的表情。
如图3所示，人脸部位图像①与③和④相邻，则人脸部位图像①具有两个邻接关系，同理可知，人脸部位图像②、③、④分别具有两个邻接关系、三个邻接关系、三个邻接关系。此时若将①作为图像序列的第一顺位图像，则图像序列前面可以为①→③→④→②→③，但是此时由于不包含①与④之间的邻接关系，此时需要最后补上“①→④”或“④→①”，即完整的图像序列可以为①→③→④→②→③→④→①，该图像序列包含七个人脸部位图像。若将具有最多邻接关系的人脸部位图像作为图像序列的第一顺位图像，即将人脸部位图像③或④作为第一顺位图像，可以减小图像序列的长度。具体的，将人脸部位图像③作为第一顺位图像，则该图像序列可以为③→①→④→②→③→④，或者③→④→①→③→②→④，此时的图像序列包含六个人脸部位图像，该图像序列在包含所有邻接关系的同时，还具有较小的长度。
其中，由于如何将人脸图像划分为多个人脸部位图像，以及人脸部位图像的邻接关系均可以是预先设置好的，故可以预先设置好生成图像序列的方式，在从待识别图像中提取出人脸图像后，可以直接按照生成图像序列的方式生成该人脸图像的图像序列。
步骤C3：基于循环神经网络将所述图像序列转换为所述人脸图像的人脸特征序列。
本申请实施例中，利用循环神经网络可以记忆相邻序列的特点，基于循环神经网络生成人脸图像的人脸特征序列。具体的，将图像序列中的每个人脸部位图像依次作为循环神经网络的输入，从而可以生成包含人脸部位图像的邻接关系的人脸特征序列。循环神经网络的一种结构示意图参见图4所示，其中，x 1至x 4表示循环神经网络的输入，E 1至E 4表示循环神经网络的输出，s 1至s 4表示循环神经网络的网络参数；若本实施例中的图像序列包含四个人脸部位图像①②③①，则将四个人脸部位图像①②③①分别作为x 1、x 2、x 3、x 4输入至该循环神经网络中，从而可以确定相应的输出，此时即可以将E 1E 2E 3E 4拼接为一个整体并作为人脸特征序列。
本申请实施例中，基于循环神经网络可以生成包含所有人脸部位图像以及 所有邻接关系的图像序列，可以保证表情识别的准确性；且将多维的人脸图像转换为一维的人脸特征序列，可以更加快速地训练表情识别模型，模型的识别结果更加准确。
此外，基于一张人脸图像可能不能正确判断学生的表情，例如学生眨眼，若拍摄装置正好捕捉到学生闭眼时的图像，此时并不能说明该学生在闭眼睡觉。本申请实施例中基于多帧连续的人脸图像来确定学生的动态特征，从而可以动态确定学生的表情信息是否异常，例如确定学生是否走神等。其中，“将人脸特征序列作为预先训练好的表情识别模型的输入，根据表情识别模型确定人脸图像所对应的表情信息”具体可以包括；
步骤D1：连续获取多帧的待识别图像，从多帧的待识别图像中依次提取出具有相同身份标识的多帧的人脸图像，并确定每一帧的人脸图像的人脸特征序列。
步骤D2：将多帧的人脸图像的人脸特征序列作为预先训练好的表情识别模型的输入，根据表情识别模型确定多帧的人脸图像所对应的表情信息。
本申请实施例中，基于多张连续拍摄的人脸图像，可以提取出学生表情的动态变化，使得判断结果更加准确。对于多张人脸图像的人脸特征序列，可以将多张人脸图像的人脸特征序列拼接为一个整体序列；或者，也可以基于另一个循环神经网络，将每个人脸特征序列依次作为循环神经网络的输入，从而生成包含该学生动态变化特征的完整特征序列。其中，该循环神经网络具体可采用LSTM(Long Short-Term Memory，长短期记忆网络)。
在上述实施例的基础上，在步骤103“生成异常身份标识的异常行为记录”之后，该方法还包括：
步骤E1：获取相关联系统中记录的异常身份标识的关联信息，根据关联信息以及异常行为记录生成异常身份标识的综合教学记录；或者
步骤E2：将异常行为记录发送至相关联系统，指示相关联系统基于相关联系统中记录的异常身份标识的关联信息以及异常行为记录生成异常身份标识的综合教学记录。
本实施例中，可以与其他的相关联系统进行对接或信息共享，该相关联系统具体可以是与教学相关的系统，例如成绩系统等，从该相关联系统中可以获取到相应的关联信息，例如学生成绩信息等；结合学生在课堂上的异常行为记录、以及成绩等信息，基于大数据分析技术可以生成该学生的综合性报告，即综合教学记录，基于该综合教学记录可以更加方便老师针对该学生进行适应性教学。
本申请实施例提供的一种基于人脸识别的监测方法，在提取待识别图像中学生的人脸图像的同时，还确定每个人脸图像对应的学生的身份标识，并基于表情识别确定学生表情是否异常；在表情异常时可以准确定位异常的学生，且可以为相应的学生生成异常行为记录。该方式可以自动识别全班学生的表情情绪变化，并抓取异常的表情，不需要老师过多参与；基于异常行为记录可以有针对性地分析相应学生的教学概况，也方便教师有针对性地进行教学，可以提 高教师的教学效果。基于循环神经网络可以生成包含所有人脸部位图像以及所有邻接关系的图像序列，可以保证表情识别的准确性；且将多维的人脸图像转换为一维的人脸特征序列，可以更加快速地训练表情识别模型，模型的识别结果更加准确。
以上详细介绍了基于人脸识别的监测方法流程，该方法也可以通过相应的装置实现，下面详细介绍该装置的结构和功能。
本申请实施例提供的一种基于人脸识别的监测装置，参见图5所示，包括：
获取模块51，用于获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；
表情识别模块52，用于确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；
异常记录模块53，用于在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
在上述实施例的基础上，所述获取模块51确定每个所述人脸图像所对应的身份标识包括：
对所述人脸图像与人脸数据库进行匹配处理，将所述人脸数据库中与所述人脸图像相匹配的人脸的身份标识作为所述人脸图像的身份标识；或者
预先设置所述待识别图像中每个图像区域与身份标识之间的对应关系，根据所述人脸图像在所述待识别图像中所属的图像区域以及图像区域与身份标识之间的对应关系确定所述人脸图像所对应的身份标识；或者
基于设在不同位置的多个拍摄装置所采集的图像生成具有景深信息的待识别图像，并提取出所述待识别图像中具有景深信息的人脸图像；根据所述人脸图像的景深信息以及所述人脸图像在所述待识别图像中的位置确定所述人脸图像在世界坐标系下的三维坐标，将所述三维坐标所对应的用户的身份标识作为所述人脸图像的身份标识。
在上述实施例的基础上，所述表情识别模块52包括：
图像划分单元，用于将所述人脸图像划分为多个部位，并确定每个部位所对应的人脸部位图像；
表情识别单元，用于根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列，并将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息。
在上述实施例的基础上，所述表情识别单元根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列包括：
根据所述人脸部位图像在所述人脸图像中的位置分别确定每个所述人脸部位图像与其他人脸部位图像之间的邻接关系；
将具有最多邻接关系的人脸部位图像作为待定的图像序列的第一顺位图像，之后将所述第一顺位图像作为当前顺位图像，并将与所述当前顺位图像具有邻接关系的其他人脸部位图像作为下一顺位图像，重复上述基于当前顺位图 像选取下一顺位图像的过程，直至遍历所有的所述人脸部位图像、且遍历所述人脸部位图像之间的所有邻接关系，最终生成包含所有的所述人脸部位图像、以及所述人脸部位图像之间的所有邻接关系的图像序列；
基于循环神经网络将所述图像序列转换为所述人脸图像的人脸特征序列。
在上述实施例的基础上，所述表情识别单元将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息包括；
连续获取多帧的待识别图像，从多帧的所述待识别图像中依次提取出具有相同身份标识的多帧的人脸图像，并确定每一帧的人脸图像的人脸特征序列；
将多帧的人脸图像的人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定多帧的所述人脸图像所对应的表情信息。
在上述实施例的基础上，所述表情识别模块52还包括训练单元；
在所述表情识别单元将所述人脸特征序列作为预先训练好的表情识别模型的输入之前，所述训练单元用于：
确定训练样本，并建立表情识别模型，所述训练样本包括人脸样本图像的人脸特征序列以及与所述人脸样本图像相对应表情信息；将所述人脸样本图像的人脸特征序列作为所述表情识别模型的输入、将与所述人脸样本图像相对应表情信息作为所述表情识别模型的输出，对所述表情识别模型进行训练，并生成训练好的表情识别模型。
在上述实施例的基础上，所述装置还包括综合处理模块；
在所述异常记录模块53生成所述异常身份标识的异常行为记录之后，所述综合处理模块用于：
获取相关联系统中记录的所述异常身份标识的关联信息，根据所述关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录；或者
将所述异常行为记录发送至相关联系统，指示所述相关联系统基于所述相关联系统中记录的所述异常身份标识的关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录。
本申请实施例提供的一种基于人脸识别的监测装置，在提取待识别图像中学生的人脸图像的同时，还确定每个人脸图像对应的学生的身份标识，并基于表情识别确定学生表情是否异常；在表情异常时可以准确定位异常的学生，且可以为相应的学生生成异常行为记录。该方式可以自动识别全班学生的表情情绪变化，并抓取异常的表情，不需要老师过多参与；基于异常行为记录可以有针对性地分析相应学生的教学概况，也方便教师有针对性地进行教学，可以提高教师的教学效果。基于循环神经网络可以生成包含所有人脸部位图像以及所有邻接关系的图像序列，可以保证表情识别的准确性；且将多维的人脸图像转换为一维的人脸特征序列，可以更加快速地训练表情识别模型，模型的识别结果更加准确。
本申请实施例还提供了一种存储有计算机可读指令的存储介质，所述存储介质为易失性存储介质或非易失性存储介质，该计算机可读指令被一个或多个 处理器执行时，使得一个或多个处理器执行以下步骤：获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
图6示出了本申请的另一个实施例的一种计算机设备的结构框图。所述计算机设备1100可以是具备计算能力的主机服务器、个人计算机PC、或者可携带的便携式计算机或终端等。本申请具体实施例并不对计算机设备的具体实现做限定。
该计算机设备1100包括至少一个处理器(processor)1110、通信接口(Communications Interface)1120、存储器(memory array)1130和总线1140。其中，处理器1110、通信接口1120、以及存储器1130通过总线1140完成相互间的通信。
通信接口1120用于与网元通信，其中网元包括例如虚拟机管理中心、共享存储等。
处理器1110用于执行程序。处理器1110可能是一个中央处理器CPU，或者是专用集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本申请实施例的一个或多个集成电路。
存储器1130用于可执行的指令。存储器1130可能包含高速RAM存储器，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1130也可以是存储器阵列。存储器1130还可能被分块，并且所述块可按一定的规则组合成虚拟卷。存储器1130存储的指令可被处理器1110执行，以使处理器1110能够执行上述任意方法实施例中的方法。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (20)

  1. 一种基于人脸识别的监测的方法，包括：
    获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；
    确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；
    在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
  2. 根据权利要求1所述的方法，所述确定每个所述人脸图像所对应的身份标识包括：
    对所述人脸图像与人脸数据库进行匹配处理，将所述人脸数据库中与所述人脸图像相匹配的人脸的身份标识作为所述人脸图像的身份标识；或者
    预先设置所述待识别图像中每个图像区域与身份标识之间的对应关系，根据所述人脸图像在所述待识别图像中所属的图像区域以及图像区域与身份标识之间的对应关系确定所述人脸图像所对应的身份标识；或者
    基于设在不同位置的多个拍摄装置所采集的图像生成具有景深信息的待识别图像，并提取出所述待识别图像中具有景深信息的人脸图像；根据所述人脸图像的景深信息以及所述人脸图像在所述待识别图像中的位置确定所述人脸图像在世界坐标系下的三维坐标，将所述三维坐标所对应的用户的身份标识作为所述人脸图像的身份标识。
  3. 根据权利要求1所述的方法，所述确定所述人脸图像中的表情信息包括：
    将所述人脸图像划分为多个部位，并确定每个部位所对应的人脸部位图像；
    根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列，并将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息。
  4. 根据权利要求3所述的方法，所述根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列包括：
    根据所述人脸部位图像在所述人脸图像中的位置分别确定每个所述人脸部位图像与其他人脸部位图像之间的邻接关系；
    将具有最多邻接关系的人脸部位图像作为待定的图像序列的第一顺位图像，之后将所述第一顺位图像作为当前顺位图像，并将与所述当前顺位图像具有邻接关系的其他人脸部位图像作为下一顺位图像，重复上述基于当前顺位图像选取下一顺位图像的过程，直至遍历所有的所述人脸部位图像、且遍历所述人脸部位图像之间的所有邻接关系，最终生成包含所有的所述人脸部位图像、以及所述人脸部位图像之间的所有邻接关系的图像序列；
    基于循环神经网络将所述图像序列转换为所述人脸图像的人脸特征序列。
  5. 根据权利要求3所述的方法，所述将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的 表情信息包括；
    连续获取多帧的待识别图像，从多帧的所述待识别图像中依次提取出具有相同身份标识的多帧的人脸图像，并确定每一帧的人脸图像的人脸特征序列；
    将多帧的人脸图像的人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定多帧的所述人脸图像所对应的表情信息。
  6. 根据权利要求3-5任一所述的方法，在所述将所述人脸特征序列作为预先训练好的表情识别模型的输入之前，还包括：
    确定训练样本，并建立表情识别模型，所述训练样本包括人脸样本图像的人脸特征序列以及与所述人脸样本图像相对应表情信息；
    将所述人脸样本图像的人脸特征序列作为所述表情识别模型的输入、将与所述人脸样本图像相对应表情信息作为所述表情识别模型的输出，对所述表情识别模型进行训练，并生成训练好的表情识别模型。
  7. 根据权利要求1-5任一所述的方法，在所述生成所述异常身份标识的异常行为记录之后，还包括：
    获取相关联系统中记录的所述异常身份标识的关联信息，根据所述关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录；或者
    将所述异常行为记录发送至相关联系统，指示所述相关联系统基于所述相关联系统中记录的所述异常身份标识的关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录。
  8. 一种基于人脸识别的监测装置，包括：
    获取模块，用于获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；
    表情识别模块，用于确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；
    异常记录模块，用于在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
  9. 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现一种基于人脸识别的监测的方法，其中，所述基于人脸识别的监测的方法包括以下步骤：
    获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；
    确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；
    在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
  10. 根据权利要求9所述的计算机可读存储介质，所述确定每个所述人脸图像所对应的身份标识包括：
    对所述人脸图像与人脸数据库进行匹配处理，将所述人脸数据库中与所述人脸图像相匹配的人脸的身份标识作为所述人脸图像的身份标识；或者
    预先设置所述待识别图像中每个图像区域与身份标识之间的对应关系，根据所述人脸图像在所述待识别图像中所属的图像区域以及图像区域与身份标识之间的对应关系确定所述人脸图像所对应的身份标识；或者
    基于设在不同位置的多个拍摄装置所采集的图像生成具有景深信息的待识别图像，并提取出所述待识别图像中具有景深信息的人脸图像；根据所述人脸图像的景深信息以及所述人脸图像在所述待识别图像中的位置确定所述人脸图像在世界坐标系下的三维坐标，将所述三维坐标所对应的用户的身份标识作为所述人脸图像的身份标识。
  11. 根据权利要求9所述的计算机可读存储介质，所述确定所述人脸图像中的表情信息包括：
    将所述人脸图像划分为多个部位，并确定每个部位所对应的人脸部位图像；
    根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列，并将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息。
  12. 根据权利要求11所述的计算机可读存储介质，所述根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列包括：
    根据所述人脸部位图像在所述人脸图像中的位置分别确定每个所述人脸部位图像与其他人脸部位图像之间的邻接关系；
    将具有最多邻接关系的人脸部位图像作为待定的图像序列的第一顺位图像，之后将所述第一顺位图像作为当前顺位图像，并将与所述当前顺位图像具有邻接关系的其他人脸部位图像作为下一顺位图像，重复上述基于当前顺位图像选取下一顺位图像的过程，直至遍历所有的所述人脸部位图像、且遍历所述人脸部位图像之间的所有邻接关系，最终生成包含所有的所述人脸部位图像、以及所述人脸部位图像之间的所有邻接关系的图像序列；
    基于循环神经网络将所述图像序列转换为所述人脸图像的人脸特征序列。
  13. 根据权利要求11所述的计算机可读存储介质，所述将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息包括；
    连续获取多帧的待识别图像，从多帧的所述待识别图像中依次提取出具有相同身份标识的多帧的人脸图像，并确定每一帧的人脸图像的人脸特征序列；
    将多帧的人脸图像的人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定多帧的所述人脸图像所对应的表情信息。
  14. 根据权利要求11-13任一所述的计算机可读存储介质，在所述将所述人脸特征序列作为预先训练好的表情识别模型的输入之前，还包括：
    确定训练样本，并建立表情识别模型，所述训练样本包括人脸样本图像的人脸特征序列以及与所述人脸样本图像相对应表情信息；
    将所述人脸样本图像的人脸特征序列作为所述表情识别模型的输入、将与所述人脸样本图像相对应表情信息作为所述表情识别模型的输出，对所述表情识别模型进行训练，并生成训练好的表情识别模型。
  15. 根据权利要求9-13任一所述的计算机可读存储介质，在所述生成所述异常身份标识的异常行为记录之后，还包括：
    获取相关联系统中记录的所述异常身份标识的关联信息，根据所述关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录；或者
    将所述异常行为记录发送至相关联系统，指示所述相关联系统基于所述相关联系统中记录的所述异常身份标识的关联信息以及所述异常行为记录生成所述异常身份标识的综合教学记录。
  16. 一种计算机设备，包括：
    一个或多个处理器；
    存储器；
    一个或多个计算机程序，其中所述一个或多个计算机程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个计算机程序配置用于执行一种基于人脸识别的监测的方法，其中，所述基于人脸识别的监测的方法包括以下步骤：
    获取待识别图像，并提取所述待识别图像中的所有人脸图像，确定每个所述人脸图像所对应的身份标识；
    确定所述人脸图像中的表情信息，并判断所述表情信息是否异常；
    在所述表情信息异常时，将与具有异常的所述表情信息的人脸图像相对应的身份标识作为异常身份标识，并生成所述异常身份标识的异常行为记录。
  17. 根据权利要求16所述的计算机设备，所述确定每个所述人脸图像所对应的身份标识包括：
    对所述人脸图像与人脸数据库进行匹配处理，将所述人脸数据库中与所述人脸图像相匹配的人脸的身份标识作为所述人脸图像的身份标识；或者
    预先设置所述待识别图像中每个图像区域与身份标识之间的对应关系，根据所述人脸图像在所述待识别图像中所属的图像区域以及图像区域与身份标识之间的对应关系确定所述人脸图像所对应的身份标识；或者
    基于设在不同位置的多个拍摄装置所采集的图像生成具有景深信息的待识别图像，并提取出所述待识别图像中具有景深信息的人脸图像；根据所述人脸图像的景深信息以及所述人脸图像在所述待识别图像中的位置确定所述人脸图像在世界坐标系下的三维坐标，将所述三维坐标所对应的用户的身份标识作为所述人脸图像的身份标识。
  18. 根据权利要求16所述的计算机设备，所述确定所述人脸图像中的表情信息包括：
    将所述人脸图像划分为多个部位，并确定每个部位所对应的人脸部位图像；
    根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列，并将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息。
  19. 根据权利要求18所述的计算机设备，所述根据所有的所述人脸部位图像生成所述人脸图像的人脸特征序列包括：
    根据所述人脸部位图像在所述人脸图像中的位置分别确定每个所述人脸部位图像与其他人脸部位图像之间的邻接关系；
    将具有最多邻接关系的人脸部位图像作为待定的图像序列的第一顺位图像，之后将所述第一顺位图像作为当前顺位图像，并将与所述当前顺位图像具有邻接关系的其他人脸部位图像作为下一顺位图像，重复上述基于当前顺位图像选取下一顺位图像的过程，直至遍历所有的所述人脸部位图像、且遍历所述人脸部位图像之间的所有邻接关系，最终生成包含所有的所述人脸部位图像、以及所述人脸部位图像之间的所有邻接关系的图像序列；
    基于循环神经网络将所述图像序列转换为所述人脸图像的人脸特征序列。
  20. 根据权利要求18所述的计算机设备，所述将所述人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定所述人脸图像所对应的表情信息包括；
    连续获取多帧的待识别图像，从多帧的所述待识别图像中依次提取出具有相同身份标识的多帧的人脸图像，并确定每一帧的人脸图像的人脸特征序列；
    将多帧的人脸图像的人脸特征序列作为预先训练好的表情识别模型的输入，根据所述表情识别模型确定多帧的所述人脸图像所对应的表情信息。
PCT/CN2020/087555 2019-09-12 2020-04-28 基于人脸识别的监测方法、装置、存储介质及计算机设备 WO2021047185A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910865404.X 2019-09-12
CN201910865404.XA CN110580470A (zh) 2019-09-12 2019-09-12 基于人脸识别的监测方法、装置、存储介质及计算机设备

Publications (1)

Publication Number Publication Date
WO2021047185A1 true WO2021047185A1 (zh) 2021-03-18

Family

ID=68811770

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/087555 WO2021047185A1 (zh) 2019-09-12 2020-04-28 基于人脸识别的监测方法、装置、存储介质及计算机设备

Country Status (2)

Country Link
CN (1) CN110580470A (zh)
WO (1) WO2021047185A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110580470A (zh) * 2019-09-12 2019-12-17 深圳壹账通智能科技有限公司 基于人脸识别的监测方法、装置、存储介质及计算机设备
CN111476685B (zh) * 2020-03-26 2023-06-20 深圳力维智联技术有限公司 行为分析方法、装置及设备
CN111985396A (zh) * 2020-08-20 2020-11-24 南京师范大学 一种孕妇情绪监测识别系统
CN112016509B (zh) * 2020-09-07 2023-09-22 中国银行股份有限公司 人员站位异常提醒方法及装置
CN112732770A (zh) * 2021-02-05 2021-04-30 嘉兴南洋职业技术学院 基于人工智能的教务管理系统及方法
CN113259734B (zh) * 2021-06-04 2023-02-03 重庆晋才富熙科技有限公司 针对交互场景的智能导播方法、装置、终端和存储介质

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170046965A1 (en) * 2015-08-12 2017-02-16 Intel Corporation Robot with awareness of users and environment for use in educational applications
CN107918755A (zh) * 2017-03-29 2018-04-17 广州思涵信息科技有限公司 一种基于人脸识别技术的实时专注度分析方法及系统
CN108073888A (zh) * 2017-08-07 2018-05-25 中国科学院深圳先进技术研究院 一种教学辅助方法及采用该方法的教学辅助系统
CN108875606A (zh) * 2018-06-01 2018-11-23 重庆大学 一种基于表情识别的课堂教学评价方法及系统
AU2017279806A1 (en) * 2017-05-29 2018-12-13 Saltor Pty Ltd Method and system for abnormality detection
CN109284713A (zh) * 2018-09-21 2019-01-29 上海健坤教育科技有限公司 一种基于摄像头采集表情数据的情绪识别分析系统
CN109344682A (zh) * 2018-08-02 2019-02-15 平安科技（深圳）有限公司 课堂监控方法、装置、计算机设备及存储介质
CN109766766A (zh) * 2018-12-18 2019-05-17 深圳壹账通智能科技有限公司 员工工作状况监控方法、装置、计算机设备和存储介质
CN109815795A (zh) * 2018-12-14 2019-05-28 深圳壹账通智能科技有限公司 基于人脸监测的课堂学生状态分析方法及装置
CN109829362A (zh) * 2018-12-18 2019-05-31 深圳壹账通智能科技有限公司 安检辅助分析方法、装置、计算机设备和存储介质
WO2019143962A1 (en) * 2018-01-19 2019-07-25 Board Of Regents, The University Of Texas System Systems and methods for evaluating individual, group, and crowd emotion engagement and attention
CN110580470A (zh) * 2019-09-12 2019-12-17 深圳壹账通智能科技有限公司 基于人脸识别的监测方法、装置、存储介质及计算机设备

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170046965A1 (en) * 2015-08-12 2017-02-16 Intel Corporation Robot with awareness of users and environment for use in educational applications
CN107918755A (zh) * 2017-03-29 2018-04-17 广州思涵信息科技有限公司 一种基于人脸识别技术的实时专注度分析方法及系统
AU2017279806A1 (en) * 2017-05-29 2018-12-13 Saltor Pty Ltd Method and system for abnormality detection
CN108073888A (zh) * 2017-08-07 2018-05-25 中国科学院深圳先进技术研究院 一种教学辅助方法及采用该方法的教学辅助系统
WO2019143962A1 (en) * 2018-01-19 2019-07-25 Board Of Regents, The University Of Texas System Systems and methods for evaluating individual, group, and crowd emotion engagement and attention
CN108875606A (zh) * 2018-06-01 2018-11-23 重庆大学 一种基于表情识别的课堂教学评价方法及系统
CN109344682A (zh) * 2018-08-02 2019-02-15 平安科技（深圳）有限公司 课堂监控方法、装置、计算机设备及存储介质
CN109284713A (zh) * 2018-09-21 2019-01-29 上海健坤教育科技有限公司 一种基于摄像头采集表情数据的情绪识别分析系统
CN109815795A (zh) * 2018-12-14 2019-05-28 深圳壹账通智能科技有限公司 基于人脸监测的课堂学生状态分析方法及装置
CN109766766A (zh) * 2018-12-18 2019-05-17 深圳壹账通智能科技有限公司 员工工作状况监控方法、装置、计算机设备和存储介质
CN109829362A (zh) * 2018-12-18 2019-05-31 深圳壹账通智能科技有限公司 安检辅助分析方法、装置、计算机设备和存储介质
CN110580470A (zh) * 2019-09-12 2019-12-17 深圳壹账通智能科技有限公司 基于人脸识别的监测方法、装置、存储介质及计算机设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113052064B (zh) * 2021-03-23 2024-04-02 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法

Also Published As

Publication number Publication date
CN110580470A (zh) 2019-12-17

Similar Documents

Publication Publication Date Title
WO2021047185A1 (zh) 基于人脸识别的监测方法、装置、存储介质及计算机设备
CN110889672B (zh) 一种基于深度学习的学生打卡及上课状态的检测系统
CN109522815B (zh) 一种专注度评估方法、装置及电子设备
WO2020082566A1 (zh) 基于生物识别的远程教学方法、装置、设备及存储介质
CN108399376A (zh) 学生课堂学习兴趣智能分析方法及系统
WO2019028592A1 (zh) 一种教学辅助方法及采用该方法的教学辅助系统
WO2021077382A1 (zh) 一种学习状态的判断方法、装置及智能机器人
CN110175534A (zh) 基于多任务级联卷积神经网络的授课辅助系统
CN110163567A (zh) 基于多任务级联卷积神经网络的课堂点名系统
CN111814587A (zh) 人体行为检测方法、教师行为检测方法及相关系统和装置
CN112883867A (zh) 一种基于图像情感分析的学生在线学习评估方法及系统
CN115205764B (zh) 基于机器视觉的在线学习专注度监测方法、系统及介质
CN112101124A (zh) 一种坐姿检测方法及装置
CN111553218A (zh) 一种基于人体姿势识别的智能医学技能教学监测系统
CN111353363A (zh) 一种教学效果检测方法及装置、电子设备
TW202008293A (zh) 教學品質監測系統及方法
CN114926889B (zh) 作业提交方法、装置、电子设备和存储介质
CN107578015B (zh) 一种基于深度学习的第一印象识别与回馈系统及方法
CN111178263B (zh) 一种实时表情分析方法及其装置
CN112949461A (zh) 一种学习状态的分析方法、装置以及电子设备
CN111666829A (zh) 多场景多主体身份行为情绪识别分析方法及智能监管系统
CN111523445B (zh) 一种基于改进的Openpose模型和面部微表情的考试行为检测方法
CN112766130A (zh) 一种课堂教学质量监测方法、系统、终端及存储介质
Ray et al. Design and implementation of affective e-learning strategy based on facial emotion recognition
Shah et al. Assessment of student attentiveness to e-learning by monitoring behavioural elements

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20862621

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.07.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20862621

Country of ref document: EP

Kind code of ref document: A1