WO2020034902A1 - 具有状态监控功能的智能桌、监控系统服务器及监控方法 - Google Patents

具有状态监控功能的智能桌、监控系统服务器及监控方法 Download PDF

Info

Publication number
WO2020034902A1
WO2020034902A1 PCT/CN2019/099957 CN2019099957W WO2020034902A1 WO 2020034902 A1 WO2020034902 A1 WO 2020034902A1 CN 2019099957 W CN2019099957 W CN 2019099957W WO 2020034902 A1 WO2020034902 A1 WO 2020034902A1
Authority
WO
WIPO (PCT)
Prior art keywords
recognition
image
state
face
client
Prior art date
Application number
PCT/CN2019/099957
Other languages
English (en)
French (fr)
Inventor
涂列群
刘亮
Original Assignee
昆山美卓智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 昆山美卓智能科技有限公司 filed Critical 昆山美卓智能科技有限公司
Priority to US17/267,501 priority Critical patent/US11836631B2/en
Publication of WO2020034902A1 publication Critical patent/WO2020034902A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B7/00—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
    • G08B7/06—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00 using electric transmission, e.g. involving audible and visible signalling through the use of sound and light sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q2209/00—Arrangements in telecontrol or telemetry systems
    • H04Q2209/20—Arrangements in telecontrol or telemetry systems using a distributed architecture

Definitions

  • the invention relates to the field of smart desks, and in particular, to a smart table with a status monitoring function, a monitoring system server, and a monitoring method.
  • the present invention provides a smart table with a status monitoring function, a monitoring system server, and a monitoring method.
  • a status monitoring function a monitoring system server
  • a monitoring method a monitoring method
  • the present invention provides a smart table with a status monitoring function, which includes a table body, an image acquisition device and an AI recognition module disposed on the table body, the image acquisition device is used to collect a user's face image, And input the collected face image into the AI recognition module, the AI recognition module is used to recognize whether the face in the input image is in a state of concentration, the AI recognition module is generated by a background server and periodically Update
  • the smart table further includes a communication module and / or a local prompting device, and the AI recognition module can send the recognition result of the concentration state to the background server and / or the first client through the communication module, or the local prompting device A recognition result that can be triggered according to the attention state of the AI recognition module is triggered.
  • the AI recognition module recognizes whether the face in the input image is in the state of concentration or not: using the symmetry of the eyeball, virtualizing the position of the eyeball center according to the spherical curvature of the eyeball in the image, and combining the pupil center in the image
  • the position of the eye is virtual from the eye rays emitted from the center of the eyeball towards the center of the pupil. Determine whether the attention is focused according to the convergence of the eye rays of the two eyes.
  • the eye rays do not have focus, or the focus of the eye rays stays within a certain area If the time exceeds the preset time range, it is determined that the attention is not focused; if the focus of the eye ray moves within the preset time range, it is determined that the attention is focused.
  • the AI recognition module is an emotional recognition neural network model that completes training, and the training method of the emotional recognition neural network model is as follows:
  • training samples including a number of facial image samples and pre-calibration results corresponding to the state of attention;
  • the emotion recognition neural network model performs self-learning according to the training sample and the completed minimized loss function until the accuracy of the recognition result reaches a preset percentage threshold, and then the trained emotion recognition neural network model is obtained.
  • the minimizing operation on the loss function includes:
  • the gradient of the loss function is obtained using the back propagation method
  • the updated weights are iterated for a preset number of times to complete the minimization of the loss function.
  • the local prompting device includes a control module and an execution module, the control module is electrically connected to the output of the AI recognition module, and the local prompting device can recognize the state of attention based on the output of the AI recognition module Results triggered include:
  • the control module receives the attention state recognition result output by the AI recognition module, and statistically recognizes the number of the state result of concentration and the state result of inattention. If the state result of concentration is proportion Below the preset proportional threshold, the control module triggers the execution module to issue a prompt signal, and the prompt signal includes an acoustic, light, and / or vibration signal.
  • the image acquisition device includes one or more cameras.
  • the smart table further includes a terminal downloaded with a second client, and the second client and the first client can perform message transmission through a background server;
  • the second client can receive a local prompt request sent by the first client, and the terminal that has downloaded the second client triggers a local prompt device to issue a prompt signal according to the local prompt request.
  • the prompt signal includes sound, light, and And / or vibration signals.
  • the present invention discloses a status monitoring system server.
  • the server receives an image uploaded by wireless communication.
  • the server includes an AI recognition module and a processor.
  • the AI recognition module is configured to identify a received image.
  • the human face is in a state of inattention or inattention
  • the processor is configured to count the recognition results of the AI recognition module
  • the server uses the wireless image to receive the received images and / or the statistical results of the processor Send to the corresponding first client.
  • the server is further configured to forward the message sent by the first client to the designated second client through wireless communication.
  • the present invention provides a state monitoring method based on emotion recognition, including the following steps:
  • the method further includes:
  • a face region in the frame image is extracted, and a corresponding face region image is generated.
  • the training method of the emotion recognition neural network model is as follows:
  • training samples including a number of facial image samples and pre-calibration results corresponding to the state of attention;
  • the emotion recognition neural network model performs self-learning according to the training sample and the completed minimized loss function until the accuracy of the recognition result reaches a preset percentage threshold, and then the trained emotion recognition neural network model is obtained.
  • the method further includes:
  • the invention also provides a working and learning state monitoring method based on face recognition technology, including a camera system for face camera, connecting a processing unit for video and image processing, and connecting a memory for data management.
  • the monitoring method includes the following: Steps: S1: Collect facial feature images or videos through the camera; S2: Extract the facial area in the video image according to the face recognition algorithm, and extract the feature points / quantities in the facial area; S3: According to the extracted face The characteristics of the area determine whether the work and learners are distracted and not in the state of study or work, that is, whether the attention is distracted;
  • the system also includes a network system, which connects the processing unit and memory, and transfers or directly stores the camera system data in the cloud of the network system;
  • step S2 extracting the face area in the video image in the working and learning state includes the following steps: S2.1: image sampling of the face video of the collected work and learning, the sampling value is 2 frames or several frames of pictures Compare feature quantities or feature points with each other, or be user-defined, and the time range is N seconds or N frames for comparison; or if multiple cameras are involved, compare the feature sets collected between different cameras;
  • S2.1 According to the face recognition principle, feature extraction is performed on the face image in the image, and the features are collected into the inventory;
  • S2.3 If multiple cameras are involved, different cameras are The collected feature sets are compared, and the number of repeated personnel is checked and the weight is eliminated according to the set threshold to prevent the repeated counting of personnel;
  • the method further includes: establishing a face library of people in monitored work and learning states, identifying each person's concentration data through face comparisons, thereby determining the concentration behavior of a particular person, and analyzing its parameters And characteristics;
  • the processor is also connected to a camera for shooting the area where the learning or work content is located;
  • the method further includes: importing status monitoring process data to form a process concentration distribution analysis of the monitored person, which specifically includes: 1. issuing a multi-level alarm signal to a higher-level administrator on a network segment according to a set threshold, especially on the network side Senior administrator; 2. Use time value as the dimension for data association; 3. Use time as the axis to form the process concentration distribution result in the monitoring scene / area to complete the monitoring of the work and learning status of the monitored person.
  • the smart table is equipped with a local AI identification module to monitor the learning status of the learner in off-grid conditions and urge the improvement of learning efficiency;
  • the communication module of the smart table is connected to the cloud server to implement cloud monitoring and remote monitoring.
  • FIG. 1 is a module block diagram of a smart table with a status monitoring function according to an embodiment of the present invention
  • FIG. 2 is a block diagram of basic modules of a status monitoring system according to an embodiment of the present invention.
  • FIG. 3 is a module block diagram of a state monitoring system with a face feature extraction function according to an embodiment of the present invention
  • FIG. 4 is a flowchart of a state monitoring method based on emotion recognition provided by an embodiment of the present invention.
  • FIG. 5 is a flowchart of steps for extracting facial features in a state monitoring method according to an embodiment of the present invention
  • FIG. 6 is a flowchart of a method for reminding and controlling a first client in a status monitoring method according to an embodiment of the present invention.
  • a smart table with a status monitoring function includes a table body, an image acquisition device and an AI recognition module provided on the table body.
  • the image acquisition device is used to collect a user's face image, and the collected face image is input to the AI recognition module, and the AI recognition module is used to recognize whether a face is in the attention focus in the input image Status, the AI recognition module is generated by the background server and updated regularly.
  • the smart table can also implement a status monitoring function even if the network is disconnected: the smart table is provided with a local prompting device, and the local prompting device includes a control module and an execution module,
  • the control module is electrically connected to the output terminal of the AI recognition module, and the control module receives the attention state recognition result output by the AI recognition module, and statistically recognizes the state result of concentration and inattention
  • the number of state results if the proportion of the state results of concentration is lower than a preset proportion threshold, the control module triggers the execution module to issue a prompt signal, the prompt signal includes sound, light and / or vibration signal.
  • the image acquisition device collects the user's face image at equal time intervals, so that the ratio of the number of pictures to be counted can be analogized to the ratio of time, for example, within 40 minutes, the image acquisition device collects 4800 Frame images, or captured video streams, can be extracted according to the sequence at an interval of 4,800 frames.
  • Each frame of image is input to the AI recognition module for recognition.
  • the recognition result of the AI recognition module is either 1 (indicating concentration). Or 0 (indicating inattention), the control module counts that the number of frame images identified as 1 among the 4800 frames of images is N, and its proportion is N / 4800, which is equivalent to the concentration of attention Time as a percentage of total time (40 minutes). According to expert analysis, it is basically impossible to maintain concentration during the entire 40 minutes.
  • the results of concentration evaluation can be formulated. For example, if the concentration is less than 25 minutes in 40 minutes, the evaluation is attention. If the concentration is not satisfactory, if the concentration time is more than or equal to 32 minutes, it is evaluated as excellent concentration, etc. According to the above standards, if the number of frame images with a statistical recognition result of 1 by the control module is less than 3000 (that is, 4800 * 25/40) frames, the control module triggers the execution module to issue a prompt signal, and the execution module may be Indicator lights, loudspeakers or vibration motors, etc., can also be combined with multiple prompts.
  • online monitoring reminders can be implemented.
  • the background server can perform model update optimization on the AI recognition module to continuously improve the model recognition. Accuracy.
  • the process of online monitoring is as follows: the AI recognition module can communicate with the background server through the communication module, and connect with the mobile client of the corresponding monitoring party (such as the parent) through the background server, and finally send the AI recognition result or video image to Analyze the AI recognition results on the parent's mobile phone or the background server (such as the concentration curve and concentration evaluation results, such as the process concentration distribution curve in the monitoring area based on the time axis) Send it to the parent's mobile phone, or send a reminder message to the second client on the smart table side to remind the monitored party to concentrate.
  • the mobile client of the monitoring party such as the parent
  • the client integrated on the smart desk or the student's own mobile phone or other terminal such as a tablet
  • the smart table of the present invention is not specifically limited to a desk, a work table, or other types of tables.
  • the first solution is that the AI recognition module recognizes whether the human face is in the state of concentration in the input image, including: using the symmetry of the eyeball, virtualizing the position of the eyeball center according to the spherical curvature of the eyeball in the image, and combining the image
  • the position of the middle pupil center is virtual from the eye rays emitted from the eyeball center toward the pupil center.
  • the AI recognition module is actually a trained emotion recognition neural network model.
  • the AI recognition module is trained by a background server.
  • the method for training the emotion recognition neural network model is as follows:
  • a training sample is obtained.
  • the training sample includes a number of facial image samples and a pre-calibration result corresponding to the state of attention.
  • the calibration of the training samples is performed by a professional psychologist or an educator, that is, for a frame of image, a calibration of 1 indicates that the face in the frame of image reflects the state of concentration, and the calibration A value of 0 indicates that the face in the frame image reflects a state of inattention.
  • the calibrated training samples are input into the neural network model for training and learning. The more training samples are, the more accurate the calibration is, and the higher the recognition accuracy of the emotion recognition neural network model after the last training is completed.
  • the loss function and minimize it are defined. Specifically, according to the weight and loss function of the neural network, the gradient of the loss function is obtained by using the back propagation method; according to the gradient, the weight of the neural network is updated by using the stochastic gradient descent method; Iterate to complete the minimization of the loss function.
  • the emotion recognition neural network model performs self-learning according to the training samples and the minimized loss function, until the accuracy of the recognition result reaches a preset percentage threshold, and then the trained emotion recognition neural network model is obtained.
  • the percentage threshold is artificially set. The higher the set value, the longer the training time and the more difficult the training is, and the greater the recognition accuracy of the completed model is.
  • the image acquisition device includes one or more cameras
  • the table body has a liftable structure
  • the camera is raised as the table top of the table body is raised or as the table The table top is lowered and lowered. That is, the height between the camera and the desktop of the table body remains unchanged, so that regardless of the height of the user of the smart table, when the desktop is adjusted to the height suitable for the user, the camera can capture the face image of the person .
  • the shooting directions are concentrated towards the same user. Compared to a single camera, setting multiple cameras can capture face images from multiple angles. Among them, the face image taken from the front is the main judgment.
  • the side image can be left unanalyzed. Only when the front image cannot be confidently judged the correct state In order to improve the accuracy of the recognition results of the emotion recognition neural network model, side-shooting images are needed for analysis.
  • the camera is preferably a large wide-angle lens, and the fixed position of the camera is variable.
  • the smart table further includes a terminal on which a second client is downloaded (the terminal may be a smart terminal for students, or a smart terminal integrated on a smart table).
  • the terminal On the premise of connecting to the network, the second client and the first client can perform message transmission through a background server, for example, the second client can perform voice and / or video communication with the first client, The voice and / or video communication may be initiated by the first client to the second client, or may be initiated by the second client to the first client.
  • the second client can receive a local prompt request sent by the first client, and the terminal downloaded with the second client triggers a local prompt device to issue a prompt signal according to the local prompt request, and the prompt signal includes a sound, Light and / or vibration signals.
  • the prompt signal includes a sound, Light and / or vibration signals.
  • the parent inquires about the user learning in front of the smart table through the first client, the parent clicks the reminder button on the first client, and the background server receives the reminder action of the first client.
  • Sending a prompt signal to a second client corresponding to the first client such as causing the second client to receive a message ringtone or controlling a terminal where the second client is located to play a prompt animation or put it in a vibration mode.
  • a status monitoring system server receives an image uploaded by wireless communication.
  • the difference between the server and the AI recognition module in the above embodiment is that it is set locally.
  • the AI recognition module in this embodiment is provided on a server or a cloud.
  • the server includes an AI recognition module and a processor, and the AI recognition module is used to recognize that a face in the received image is in a state of concentration or inattention.
  • the processor is configured to count the recognition results of the AI recognition module, and the server sends the received images and / or the statistical results of the processor to a corresponding first client through wireless communication.
  • the state monitoring system is shown in FIG.
  • the processor is further configured to extract a face region in the frame image according to a face recognition algorithm, and generate a corresponding person. Face area image.
  • the AI recognition module is set on the server side. Since the server corresponds to image acquisition devices of multiple smart tables, the server can receive image resources transmitted from multiple smart tables, so that the server-side AI recognition module can (compared to the setting In the case of the smart table, it has a wider range of learning material resources to better adaptive learning, improve model accuracy and recognition accuracy.
  • the status monitoring system server is further configured to forward a message sent by the first client to a designated second client through wireless communication.
  • the message includes, for example, an audio / video call request, such as a reminder to focus attention.
  • the signal, or a trigger signal that triggers a local prompt device on the smart table is as follows:
  • the server may The reminder signal is sent to the second client.
  • the control module can control the smart reminder signal of the first client.
  • the execution module in the local prompting device sends a local prompt signal.
  • the execution module is a vibration motor embedded in the desktop to achieve a preset time of vibration, that is, remote monitoring by parents.
  • a vibration motor embedded in the desktop to achieve a preset time of vibration, that is, remote monitoring by parents.
  • you press the mobile client button you can trigger the vibration of the desktop of the smart table, To remind the monitored subject to concentrate, this is very effective for users who are not attentive to concentrate immediately.
  • a state monitoring method based on emotion recognition is provided.
  • the monitoring method includes the following steps:
  • S11 Obtain video information of an object to be monitored, and extract one or more frame images at equal time intervals according to the video.
  • the video information is captured by an image acquisition device (camera device) on a smart table, and an image of a face area including a monitored object can be captured.
  • an image acquisition device camera device
  • the frame images are sequentially input into a pre-trained emotional recognition neural network model, where the emotional recognition neural network model is used to recognize that a face in the frame image is in a state of concentration or inattention.
  • S15 may also be executed to determine whether the proportion of the results of the state of concentration is lower than the preset proportion threshold. If so, execute S16, send a reminder message to the first client that implements the monitoring party, or directly to The second client of the monitored party sends a prompt message, as shown in FIG. 4.
  • the status monitoring method further includes: receiving a message or request sent by a first client that implements the monitoring party; and forwarding the message or request to a second client of the monitored party,
  • the message or request includes, for example, an audio / video call request, a signal such as a reminder to concentrate, or a trigger signal that triggers a local prompt device on the smart table.
  • the training method of the emotion recognition neural network model in S12 is as follows:
  • training samples including a number of facial image samples and pre-calibration results corresponding to the state of attention;
  • the emotion recognition neural network model performs self-learning according to the training sample and the completed minimized loss function until the accuracy of the recognition result reaches a preset percentage threshold, and then the trained emotion recognition neural network model is obtained.
  • the state monitoring method based on emotion recognition is shown in FIG. 5 and includes the following steps:
  • S21 Obtain video information of an object to be monitored, and extract one or more frame images of the video at equal time intervals according to a time sequence.
  • the video information is captured by an image acquisition device (camera device) on a smart table, and an image of a face area including a monitored object can be captured.
  • an image acquisition device camera device
  • the frame images are sequentially input into a pre-trained emotional recognition neural network model, where the emotional recognition neural network model is used to recognize that a face in the frame image is in a state of concentration or inattention.
  • the state monitoring method based on emotion recognition may further include the following steps, as shown in FIG. 6:
  • the second client After the second client receives the reminder request, one way is to send a reminder message on the second client or its terminal to remind the monitored party to concentrate, and the other way is to use the terminal where the second client is located Sending the reminder request to the control module of the local prompting device, and the control module controls the execution module of the local prompting device to perform sound, light, and / or vibration alarm operations.
  • the invention discloses a work and learning state monitoring system and system based on face recognition technology, and particularly relates to a work and learning state monitoring system based on face recognition technology for supervising students and improving learning efficiency and staff efficiency. And system.
  • the analysis of the learners is recorded directly by the supervised person, and the manual method is biased towards the subjective impression of the lecturer.
  • This technology uses a relatively new technology, which is performed by face recognition. Analyze and judge. Thus changing the way that relies on a large number of human participation and lacks practical application effects.
  • this method lacks scientificity in calculation and analysis after data collection; in actual application, it lacks correlation analysis of data application and is systematically lacking.
  • the designer actively researches and innovates in order to create a space environment optimization device and system, which makes it more convenient and practical and industrially valuable.
  • the primary object of the present invention is to provide a working and learning state monitoring system based on face recognition technology, which provides objective and true data results for the academic analysis of the teaching or the monitoring of work efficiency through scientific big data algorithms.
  • a further object of the present invention is to provide a working and learning state monitoring system based on face recognition technology.
  • the system can provide an efficient learning environment and an efficient working environment, and can calculate the optimal learning time and working time period of the monitored person.
  • the technical solution of the present invention is as follows: In one embodiment of the present invention, a method for monitoring work and learning status based on face recognition technology is provided.
  • the monitoring method includes the following steps: S1: Collect a facial image or video of a person's face through a camera; S2: Extract a person in the video image according to a face recognition algorithm Face area, extract feature points / quantities in the face area; S3: Determine whether the work and learners are distracted based on the features of the extracted face area, whether they are not studying or working, that is, whether the attention is distracted.
  • the system further comprises a network system, which connects the processing unit and the memory, and transfers or directly stores the camera system data in the cloud of the network system;
  • step S2 extracting the face area in the video image in the working and learning state includes the following steps: S2.1: image sampling of the face video of the collected work and learning, the sampling value is 2 frames Or several frames of pictures comparing feature quantities or feature points with each other, or user-defined, the time range is compared in N seconds or N frames; or if multiple cameras are involved, the feature set collected between different cameras is performed Compared;
  • S2.2 According to the face recognition principle, feature extraction is performed on the face image in the image, and the features are collected into the inventory;
  • S2.3 If multiple cameras are involved, the The feature sets collected between different cameras are compared, and the number of repeated personnel, weight reduction is confirmed according to the set threshold, and the personnel is not counted repeatedly;
  • the method further includes: establishing a face library of people in the monitored working and learning states, and comparing each person's concentration data through face comparison to determine the concentration behavior of a specific person , Analyze its parameters and characteristics;
  • the processor is further connected to a camera for capturing the area where the study or work content is located; for example, the computer screen of a working tool, a desktop book during study, etc., generally the camera is set in front of the face of the monitored object It is preferred that its setting position is adjustable.
  • the method further includes: importing status monitoring process data to form a process concentration distribution analysis of the monitored person, which specifically includes: 1. issuing a multi-level alarm signal to a superior administrator in accordance with a set threshold in a network segment, Especially the senior administrators on the network side; 2. Associate data with the time value as the dimension; 3. Use time as the axis to form the distribution result of process concentration in the monitoring scene / area to complete the monitoring of the work and learning status of the monitored person.
  • the smart table of the present invention implements monitoring of attention during learning / work based on the AI recognition technology, and makes prompt actions when inattention is detected to urge concentration and improve learning / work efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Geometry (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种具有状态监控功能的智能桌、监控系统服务器及监控方法，智能桌包括桌体、设置在桌体上的图像采集装置和AI识别模块，所述图像采集装置用于采集使用者的人脸图像，并将采集到的人脸图像输入AI识别模块，AI识别模块用于识别输入的图像中人脸是否处于注意力集中状态；所述智能桌还包括通信模块和/或本地提示装置，AI识别模块能够将注意力集中状态识别结果通过通信模块发送至后台服务器和/或第一客户端，或者，本地提示装置能够根据AI识别模块的注意力集中状态识别结果受到触发。本发明的智能桌基于AI识别技术实现对学习/工作中注意力进行监测，并在监测到注意力不集中时作出提示动作，以督促集中注意力，提高学习/工作效率。

Description

具有状态监控功能的智能桌、监控系统服务器及监控方法
相关申请的交叉引用
本申请要求2018年8月11日提交的申请号为201810911166.7的专利申请的优先权。
技术领域
本发明涉及智能书桌领域，尤其涉及一种具有状态监控功能的智能桌、监控系统服务器及监控方法。
背景技术
目前市面上的智能书桌集中在智能调节桌面、智能调光等方面作出改进，但是目前的智能书桌无法对伏案学习或者工作的人进行学习状态/工作状态的监督。因此，为了督促学生集中注意力，通常需要家长在旁监督提醒。
但是这种监督方式不仅花费家长的精力，而且使学生在学习的时候倍感压力而反感这种在旁监督的方式，反而不利于注意力的集中。
现有技术中缺少一种合理、有效的学习/工作状态监控系统和方法。
发明内容
为了解决现有技术的问题，本发明提供了一种具有状态监控功能的智能桌、监控系统服务器及监控方法，鉴于对上述过程的长期观察和思考，本设计人积极加以研究创新，以期创设一种空间环境优化装置及系统，使其更具有方便实用特点和产业上价值。所述技术方案如下：
一方面，本发明提供了一种具有状态监控功能的智能桌，包括桌体、设置在桌体上的图像采集装置和AI识别模块，所述图像采集装置用于采集使用者的人脸图像，并将采集到的所述人脸图像输入所述AI识别模块，所述AI识别模块用于识别输入的图像中人脸是否处于注意力集中状态，所述AI识别模块为由 后台服务器生成并定期更新；
所述智能桌还包括通信模块和/或本地提示装置，所述AI识别模块能够将注意力集中状态识别结果通过通信模块发送至后台服务器和/或第一客户端，或者，所述本地提示装置能够根据所述AI识别模块的注意力集中状态识别结果受到触发。
进一步地，所述AI识别模块识别输入的图像中人脸是否处于注意力集中状态包括：利用眼球的对称性，根据图像中眼珠的球面弧度虚拟出眼球球心的位置，并结合图像中瞳孔中心的位置，虚拟出自眼球球心向瞳孔中心射出的目光射线，根据两眼目光射线汇聚的情况判断注意力是否集中，若所述目光射线没有焦点，或者所述目光射线的焦点在一定区域内停留的时间超过预设的时间范围，则判定为注意力不集中；若所述目光射线的焦点在预设的时间范围内发生移动，则判定为注意力集中。
进一步地，所述AI识别模块为完成训练的情绪识别神经网络模型，所述情绪识别神经网络模型的训练方法如下：
获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果；
定义损失函数，并对损失函数进行最小化操作；
所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。
进一步地，所述对损失函数进行最小化操作包括：
根据神经网络的所有权重和损失函数，采用反向传播法得到损失函数的梯度；
根据所述梯度，采用随机梯度下降法，更新神经网络的权重；
将更新的权重进行预设次数的迭代，完成对损失函数的最小化操作。
进一步地，所述本地提示装置包括控制模块与执行模块，所述控制模块与所述AI识别模块的输出端电连接，所述本地提示装置能够根据所述AI识别模块输出的注意力集中状态识别结果受到触发包括：
所述控制模块接收所述AI识别模块输出的注意力集中状态识别结果，并统 计识别得到注意力集中的状态结果与注意力不集中的状态结果的数量，若注意力集中的状态结果所占比例低于预设的比例阈值，则所述控制模块触发所述执行模块发出提示信号，所述提示信号包括声、光和/或振动信号。
可选地，所述图像采集装置包括一个或多个摄像头。
进一步地，所述智能桌还包括下载有第二客户端的终端，所述第二客户端与所述第一客户端能够通过后台服务器进行消息传输；
所述第二客户端能够接收第一客户端发送的本地提示请求，下载有所述第二客户端的终端根据所述本地提示请求触发本地提示装置发出提示信号，所述提示信号包括声、光和/或振动信号。
另一方面，本发明公开了一种状态监控系统服务器，所述服务器接收通过无线通信方式上传的图像，所述服务器包括AI识别模块及处理器，所述AI识别模块用于识别接收的图像中人脸处于注意力集中状态或注意力不集中状态，所述处理器用于统计所述AI识别模块的识别结果，所述服务器将接收的图像和/或所述处理器的统计结果通过无线通信方式发送至相应的第一客户端。
进一步地，所述服务器还用于将第一客户端发出的消息通过无线通信的方式转发给指定的第二客户端。
再一方面，本发明提供了一种基于情绪识别的状态监控方法，包括以下步骤：
获取待监控对象的视频信息，并将视频按时序提取一个或多个帧图像；
将所述帧图像顺序输入预训练的情绪识别神经网络模型，所述情绪识别神经网络模型用于识别帧图像中人脸处于注意力集中状态或注意力不集中状态；
接收所述情绪识别神经网络模型输出的对应于每个帧图像的识别结果；
统计识别得到注意力集中的状态结果所占比例。
进一步地，在将视频按时序提取一个或多个帧图像之后还包括：
根据人脸识别算法，提取所述帧图像中的人脸区域，生成对应的人脸区域图像。
进一步地，所述情绪识别神经网络模型的训练方法如下：
获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果；
定义损失函数，并对损失函数进行最小化操作；
所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。
进一步地，所述统计识别得到注意力集中的状态结果所占比例之后还包括：
将统计结果发送至施行监控方的第一客户端；或者，
若所述注意力集中的状态结果所占比例低于预设的比例阈值，则向施行监控方的第一客户端发送提示消息，或者向被监控方的第二客户端发送提示消息；或者，
接收施行监控方的第一客户端发送的消息或请求，并将所述消息或请求转发至被监控方的第二客户端。
本发明还提供了一种基于人脸识别技术的工作、学习状态监测方法，包括面向人脸摄像的摄像系统，连接用于视频、图像处理的处理单元、连接数据管理的存储器，监测方法包括以下步骤：S1：通过摄像头采集人面部特征图像或视频；S2：依据人脸识别算法，提取出视频图像中的人脸区域，提取人脸区域内的特征点/量；S3：根据提取的人脸区域的特征判断工作、学习人员是否有分心，不在学习或工作状态，即注意力是否分散；
系统还包括网络系统，连接处理单元和存储器，转存或直接在网络系统的云端存储摄像系统资料；
步骤S2中，提取工作、学习状态中的视频图像中的人脸区域包括以下步骤：S2.1：对采集的工作、学习时的脸部视频进行图像取样，取样值为2帧或若干帧图片相互比较特征量或特征点，或者由用户自定义，时间范围在N秒或N帧进行比对；或者如果涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行对比；
S2.1之后S2.2：依据人脸识别原理，对图像中的人脸图像进行特征提取，将特征集入库存取；S2.3：如涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行比对，按照设定的阈值确认人员重复数量、消重、避免人员被重复统计；
所述方法还包括：建立受监控的工作、学习状态中的人的人脸图库，通过 人脸比对从而识别每个人的专注度数据，从而判断某个特定人的专注度行为，分析其参数和特征；
所述处理器还连接一摄像头，用于拍摄学习或工作内容所在的区域；
所述方法还包括：导入状态监控过程数据，形成被监控人的过程专注度分布分析，具体包括：1、在网络段按照设定的阈值发布多级报警信号给上级管理员，特别是网络端的高级管理员；2、以时间值为维度进行数据关联；3、以时间为轴线，形成监控场景/区域内的过程专注度分布结果，完成被监控人工作、学习状态的监控。
本发明提供的技术方案带来的有益效果如下：
a.智能桌配置本地AI识别模块，实现离网状况下对学习者的学习状态进行监控，督促提高学习效率；
b.智能桌的通信模块连接云端服务器，实现云端监控和远程监控。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的具有状态监控功能的智能桌的模块框图；
图2是本发明实施例提供的状态监控系统的基本模块框图；
图3是本发明实施例提供的具有人脸特征提取功能的状态监控系统的模块框图；
图4是本发明实施例提供的基于情绪识别的状态监控方法的流程图；
图5是本发明实施例提供的状态监控方法中设有人脸特征提取步骤的流程图；
图6是本发明实施例提供的状态监控方法中第一客户端进行提醒控制的方法流程图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、装置、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其他步骤或单元。
在本发明的一个实施例中，提供了一种具有状态监控功能的智能桌，如图1所示，所述智能桌包括桌体、设置在桌体上的图像采集装置和AI识别模块，所述图像采集装置用于采集使用者的人脸图像，并将采集到的所述人脸图像输入所述AI识别模块，所述AI识别模块用于识别输入的图像中人脸是否处于注意力集中状态，所述AI识别模块由所述后台服务器生成并定期更新。
在所述AI识别模块完成训练的情况下，即使断网，所述智能桌也可以实现状态监控功能：所述智能桌上设有本地提示装置，所述本地提示装置包括控制模块与执行模块，所述控制模块与所述AI识别模块的输出端电连接，所述控制模块接收所述AI识别模块输出的注意力集中状态识别结果，并统计识别得到注意力集中的状态结果与注意力不集中的状态结果的数量，若注意力集中的状态结果所占比例低于预设的比例阈值，则所述控制模块触发所述执行模块发出提示信号，所述提示信号包括声、光和/或振动信号。优选地，所述图像采集装置等时间间隔地采集使用者的人脸图像，这样，使得统计的图片数量比值就可以类比于时间的比值，比如在40分钟内，所述图像采集装置采集了4800帧图像，或者拍摄得到的视频流可以按时序提取等时间间隔的4800帧图像，每一帧图像输入AI识别模块进行识别，所述AI识别模块的识别结果要么为1(表示注意力 集中)，要么为0(表示注意力不集中)，所述控制模块统计在这4800帧图像中，识别为1的帧图像的数量为N，则其所占比例为N/4800，相当于注意力集中的时间占总时间(40分钟)的比例。根据专家分析，全程40分钟要保持注意力集中是基本上不可能的，按照专家的建议，可以制定注意力集中评价结果，比如40分钟内注意力集中的时间少于25分钟，则评价为注意力集中度不合格，注意力集中的时间多于或等于32分钟，则评价为注意力集中度优秀，等等。按照上述标准，若所述控制模块统计识别结果为1的帧图像的数量少于3000(即4800*25/40)帧，则所述控制模块触发执行模块发出提示信号，所述执行模块可以是指示灯、扬声器喇叭或者是振动电机等等，也可以是结合多种提示方式。
除了断网状态下进行监控提醒，在本发明的一个实施例中，可以实现在线监控提醒，在线状态下，所述后台服务器可以对所述AI识别模块进行模型更新优化，以不断提高模型识别的准确率。在线监控的过程如下：所述AI识别模块能够通过通信模块与后台服务器通信连接，并通过后台服务器与对应的监控方(比如家长)的手机客户端连接，最终将AI识别结果或者视频图像发送至家长的手机上，或者经过后台服务器对AI识别结果进行分析处理后，得到的分析结果(比如注意力集中曲线及集中度评价结果，比如以时间为轴线形成监控区域内的过程专注度分布曲线)发送至家长的手机上，或者将提醒消息发送至智能桌侧的第二客户端，以提醒被监控方集中注意力。下文中将监控方比如家长的手机客户端称为第一客户端，将集成在智能桌桌体上的或者学生自己的手机(或平板等其他终端)上的客户端称为第二客户端。需要说明的是，本发明的智能桌对于具体为书桌、工作桌还是其他类型的桌子不作特别限定。
下面对AI识别模块识别输入的图像中人脸是否处于注意力集中状态的具体方案进行说明：
第一种方案是：所述AI识别模块识别输入的图像中人脸是否处于注意力集中状态包括：利用眼球的对称性，根据图像中眼珠的球面弧度虚拟出眼球球心的位置，并结合图像中瞳孔中心的位置，虚拟出自眼球球心向瞳孔中心射出的目光射线，根据两眼目光射线汇聚的情况判断注意力是否集中，若所述目光射线没有焦点，或者所述目光射线的焦点在一定区域内停留的时间超过预设的时间范围，则判定为注意力不集中；若所述目光射线的焦点在预设的时间范围内 发生移动，则判定为注意力集中。其中，针对所述目光射线没有焦点的情况，根据单帧图像即可判断出，而针对目光射线有焦点的情况，需要根据前后多帧图像才可以判断出注意力是否集中。
第二种方案，所述AI识别模块实际上为完成训练的情绪识别神经网络模型，所述AI识别模块由后台服务器训练得到，所述情绪识别神经网络模型的训练方法如下：
首先，获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果。具体地，所述训练样本的标定由专业的心理学家或教育学家进行标定，即对于某一帧图像，标定为1则表明该帧图像中的人脸反映出注意力集中的状态，标定为0则表明该帧图像中的人脸反映出注意力不集中的状态。完成标定的训练样本输入神经网络模型进行训练学习，训练样本越多，标定越准确，则最后训练完成后的情绪识别神经网络模型的识别准确性就越高。
其次，定义损失函数，并对损失函数进行最小化操作。具体地，根据神经网络的所有权重和损失函数，采用反向传播法得到损失函数的梯度；根据所述梯度，采用随机梯度下降法，更新神经网络的权重；将更新的权重进行预设次数的迭代，完成对损失函数的最小化操作。
再次，所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。其中，所述百分比阈值是人为设定的，设定值越高，则训练时间越长及训练难度越大，完成训练的模型的识别准确性也越大。
在本发明的一个优选实施例中，所述图像采集装置包括一个或多个摄像头，所述桌体具有可升降结构，所述摄像头随所述桌体桌面的升高而升高或者随所述桌体桌面的降低而降低。即所述摄像头与所述桌体桌面之间的高度保持不变，这样可以使得不管智能桌的使用者身高如何，当桌面调节到适合其的高度时，所述摄像头可以拍摄到其人脸图像。至于多个摄像头，其拍摄的方向均集中朝向同一个使用者，相比于单个摄像头，设置多个摄像头能够多个角度地拍摄人脸图像，其中，优选以正面拍摄的人脸图像为主要判断依据，侧面拍摄的图像为辅助判断依据，在正面图像能够较高准确度地判定注意力集中状态的前提下， 侧拍图像可以不加以分析，只有当正面图像无法有大的把握判定正确的状态时，需要侧拍图像辅助分析，以提高情绪识别神经网络模型的识别结果的正确率。所述摄像头优选为大广角镜头，所述摄像头固定的位置可变。
在本发明的一个优选实施例中，所述智能桌还包括下载有第二客户端的终端(该终端可以为学生的智能终端，也可以为集成在智能桌上的智能终端)，当所述终端连接网络的前提下，所述第二客户端与所述第一客户端能够通过后台服务器进行消息传输，比如所述第二客户端能够与所述第一客户端进行语音和/或视频通讯，所述语音和/或视频通讯可以由第一客户端向第二客户端发起，也可以由第二客户端向第一客户端发起。或者，所述第二客户端能够接收第一客户端发送的本地提示请求，下载有所述第二客户端的终端根据所述本地提示请求触发本地提示装置发出提示信号，所述提示信号包括声、光和/或振动信号，比如，家长通过第一客户端查询到智能桌前学习的用户注意力不集中，则家长在第一客户端点击提醒按钮，后台服务器接收到该第一客户端的提醒动作，则向与第一客户端对应的第二客户端发送提示信号，比如使第二客户端接收到消息铃声或者控制所述第二客户端所在的终端播放提示动画或使其处于振动模式。
在本发明的一个实施例中，提供了一种状态监控系统服务器，如图2所示，所述服务器接收通过无线通信方式上传的图像，与上述实施例中AI识别模块设置在本地不同的是，本实施例中的AI识别模块设置在服务器端或云端，所述服务器包括AI识别模块及处理器，所述AI识别模块用于识别接收的图像中人脸处于注意力集中状态或注意力不集中状态，所述处理器用于统计所述AI识别模块的识别结果，所述服务器将接收的图像和/或所述处理器的统计结果通过无线通信方式发送至相应的第一客户端。在本发明的一个优选实施例中，所述状态监控系统如图3所示，其中所述处理器还用于根据人脸识别算法，提取所述帧图像中的人脸区域，生成对应的人脸区域图像，这样由于提取了人脸区域的图像，而放弃了人脸区域以外的图像，可以使得降低原图像的干扰，提高最终AI识别模块的识别准确率。另外，将AI识别模块设置在服务器端，由于服务器对应于多个智能桌的图像采集装置，因此，服务器可以接收多个智能桌上传的图像资源，使得服务器端的AI识别模块能够(相比于设置在智能桌本地的情况) 具有更广泛的学习素材资源，以更好地自适应学习，提高模型精度和识别准确率。
本发明实施例的状态监控系统服务器还用于将第一客户端发出的消息通过无线通信的方式转发给指定的第二客户端，所述消息包括比如音频/视频通话请求、比如提醒集中注意力的信号，又或者是触发智能桌上的本地提示装置的触发信号，作为一种可实施例的技术方案，具体如下：当家长在第一客户端上发送智能提醒信号，所述服务器可以将该提醒信号发送至第二客户端，在第二客户端所在终端连接智能桌上的本地提示装置中的控制模块的输入端的前提下，所述控制模块可以根据所述第一客户端的智能提醒信号控制所述本地提示装置中的执行模块发出本地提示信号，比如所述执行模块为嵌置在桌面中的振动电机，实现预设时间的振动，即家长远程监控，当发现被监测对象注意力不集中的时候，按下手机客户端按钮，即可触发智能桌的桌面发生振动，以提醒被监测对象集中注意力，这对于注意力不集中的用户立马集中注意力非常有效。
在本发明的一个实施例中，提供了一种基于情绪识别的状态监控方法，参见图4，所述监控方法包括以下步骤：
S11、获取待监控对象的视频信息，并将视频按时序提取等时间间隔的一个或多个帧图像。具体地，所述视频信息由智能桌上的图像采集装置(摄像装置)摄取，能够摄取到包含被监控对象的人脸区域的图像。
S12、将所述帧图像顺序输入预训练的情绪识别神经网络模型，所述情绪识别神经网络模型用于识别帧图像中人脸处于注意力集中状态或注意力不集中状态。
S13、接收所述情绪识别神经网络模型输出的对应于每个帧图像的识别结果。
S14、统计识别得到注意力集中的状态结果所占比例。
在S14之后，一种是可以主动地将统计结果发送至施行监控方的第一客户端；另一种是等第一客户端发送查询请求之后，再将统计结果发送至施行监控方的第一客户端。
在S14之后，还可以执行S15、判断注意力集中的状态结果所占比例是否低于预设的比例阈值，若是，则执行S16、向施行监控方的第一客户端发送提示消 息，或者直接向被监控方的第二客户端发送提示消息，如图4所示。
在本发明的一个优选实施例中，所述状态监控方法还包括：接收施行监控方的第一客户端发送的消息或请求；将所述消息或请求转发至被监控方的第二客户端，所述消息或请求包括比如音频/视频通话请求、比如提醒集中注意力的信号，又或者是触发智能桌上的本地提示装置的触发信号。
S12中的所述情绪识别神经网络模型的训练方法如下：
获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果；
定义损失函数，并对损失函数进行最小化操作；
所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。
具体参见上述智能桌实施例，在此不再赘述。
在本发明的一个更优选实施例中，所述基于情绪识别的状态监控方法如图5所示，包括以下步骤：
S21、获取待监控对象的视频信息，并将视频按时序提取等时间间隔的一个或多个帧图像。具体地，所述视频信息由智能桌上的图像采集装置(摄像装置)摄取，能够摄取到包含被监控对象的人脸区域的图像。
S22、根据人脸识别算法，提取所述帧图像中的人脸区域，生成对应的人脸区域的新的帧图像。
S23、将所述帧图像顺序输入预训练的情绪识别神经网络模型，所述情绪识别神经网络模型用于识别帧图像中人脸处于注意力集中状态或注意力不集中状态。
S24、接收所述情绪识别神经网络模型输出的对应于每个帧图像的识别结果。
S25、统计识别得到注意力集中的状态结果所占比例。
所述基于情绪识别的状态监控方法还可以包括以下步骤，如图6所示：
S31、接收施行监控方的第一客户端的提醒请求；
S32、将所述提醒请求转发至被监控方的第二客户端。
第二客户端在接收到提醒请求后，一种方式是在第二客户端上或者其所在终端上发出提示信息，提醒被监控方集中注意力，另一种方式是通过第二客户端所在终端将所述提醒请求发送至本地提示装置的控制模块，由控制模块控制所述本地提示装置的执行模块执行声、光和/或振动报警操作。
本发明公开了一种基于人脸识别技术的工作、学习状态监测系统及系统，特别涉及监督学生的及提高学习效率和工作人员效率用的一种基于人脸识别技术的工作、学习状态监测系统及系统。现有的技术方案中，对于学习的人的分析，直接通过监督的人旁观的方式去记录，偏重于听课人员的主观印象的人工方式；本技术采用比较新的技术，通过人脸识别来进行分析判断。从而改变依赖大量的人力参与，缺乏实际应用效果的方式。而目前市场上出现本方法在数据采集之后的计算分析上，缺乏科学性；在实际应用上，缺乏数据应用的关联分析，系统性缺失。鉴于对上述过程的长期观察和思考，本设计人积极加以研究创新，以期创设一种空间环境优化装置及系统，使其更具有方便实用特点和产业上价值。
本发明的首要目的是提供一种基于人脸识别技术的工作、学习状态监测系统，通过科学的大数据算法，为教学的学情分析或工作效率监控，提供了客观、真实的数据结果。本发明的进一步目的是供一种基于人脸识别技术的的工作、学习状态监测系统。本系统能够提供高效的学习环境和高效的工作环境，统计出被监控人的最佳学习时间和工作状态时间段。为解决上述技术问题，本发明的技术方案如下：在本发明的一个实施例中，提供了一种基于人脸识别技术的工作、学习状态监测方法，包括面向人脸摄像的摄像系统，连接用于视频、图像处理的处理单元、连接数据管理的存储器，所述监测方法包括以下步骤：S1：通过摄像头采集人面部特征图像或视频；S2：依据人脸识别算法，提取出视频图像中的人脸区域，提取人脸区域内的特征点/量；S3：根据提取的人脸区域的特征判断工作、学习人员是否有分心，不在学习或工作状态，即注意力是否分散。
较佳地，所述系统还包括网络系统，连接处理单元和存储器，转存或直接在网络系统的云端存储摄像系统资料；
较佳地，步骤S2中，提取工作、学习状态中的视频图像中的人脸区域包括 以下步骤：S2.1：对采集的工作、学习时的脸部视频进行图像取样，取样值为2帧或若干帧图片相互比较特征量或特征点，或者由用户自定义，时间范围在N秒或N帧进行比对；或者如果涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行对比；
较佳地，S2.1之后S2.2：依据人脸识别原理，对图像中的人脸图像进行特征提取，将特征集入库存取；S2.3：如涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行比对，按照设定的阈值确认人员重复数量、消重、避免人员被重复统计；
较佳地，所述方法还包括：建立受监控的工作、学习状态中的人的人脸图库，通过人脸比对从而识别每个人的专注度数据，从而判断某个特定人的专注度行为，分析其参数和特征；
较佳地，所述处理器还连接一摄像头，用于拍摄学习或工作内容所在的区域；例如工作工具的电脑屏幕、学习时的桌面上书等，一般该摄像头设置在被监控对象的人脸前方，优选其设置位置可调。
较佳地，所述方法还包括：导入状态监控过程数据，形成被监控人的过程专注度分布分析，具体包括：1、在网络段按照设定的阈值发布多级报警信号给上级管理员，特别是网络端的高级管理员；2、以时间值为维度进行数据关联；3、以时间为轴线，形成监控场景/区域内的过程专注度分布结果，完成被监控人工作、学习状态的监控。
本发明的智能桌基于AI识别技术实现对学习/工作中注意力进行监测，并在监测到注意力不集中时作出提示动作，以督促集中注意力，提高学习/工作效率。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (12)

  1. 一种基于人脸识别技术的工作、学习状态监测方法，包括面向人脸摄像的摄像系统，连接用于视频、图像处理的处理单元、连接数据管理的存储器，其特征在于，包括以下步骤：S1：通过摄像头采集人面部特征图像或视频；S2：依据人脸识别算法，提取出视频图像中的人脸区域，提取人脸区域内的特征点/量；S3：根据提取的人脸区域的特征判断工作、学习人员是否有分心，不在学习或工作状态，即注意力是否分散；
    系统还包括网络系统，连接处理单元和存储器，转存或直接在网络系统的云端存储摄像系统资料；
    步骤S2中，提取工作、学习状态中的视频图像中的人脸区域包括以下步骤：S2.1：对采集的工作、学习时的脸部视频进行图像取样，取样值为2帧或若干帧图片相互比较特征量或特征点，或者由用户自定义，时间范围在N秒或N帧进行比对；或者如果涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行对比；
    S2.1之后S2.2：依据人脸识别原理，对图像中的人脸图像进行特征提取，将特征集入库存取；S2.3：如涉及到多个摄像头，则将不同摄像头之间采集到的特征集进行比对，按照设定的阈值确认人员重复数量、消重、避免人员被重复统计；
    所述方法还包括：建立受监控的工作、学习状态中的人的人脸图库，通过人脸比对从而识别每个人的专注度数据，从而判断某个特定人的专注度行为，分析其参数和特征；
    所述处理器还连接一摄像头，用于拍摄学习或工作内容所在的区域；
    所述方法还包括：导入状态监控过程数据，形成被监控人的过程专注度分布分析，具体包括：1、在网络段按照设定的阈值发布多级报警信号给上级管理员，特别是网络端的高级管理员；2、以时间值为维度进行数据关联；3、以时间为轴线，形成监控场景/区域内的过程专注度分布结果，完成被监控人工作、学习状态的监控。
  2. 一种具有状态监控功能的智能桌，其特征在于，包括桌体、设置在桌体上的图像采集装置和AI识别模块，所述图像采集装置用于采集使用者的人脸图像，并将采集到的所述人脸图像输入所述AI识别模块，所述AI识别模块用于识别输入的图像中人脸是否处于注意力集中状态，所述AI识别模块为由后台服务器生成并定期更新；
    所述智能桌还包括通信模块和/或本地提示装置，所述AI识别模块能够将注意力集中状态识别结果通过通信模块发送至后台服务器和/或第一客户端，或者，所述本地提示装置能够根据所述AI识别模块的注意力集中状态识别结果受到触发。
  3. 根据权利要求2所述的智能桌，其特征在于，所述AI识别模块识别输入的图像中人脸是否处于注意力集中状态包括：利用眼球的对称性，根据图像中眼珠的球面弧度虚拟出眼球球心的位置，并结合图像中瞳孔中心的位置，虚拟出自眼球球心向瞳孔中心射出的目光射线，根据两眼目光射线汇聚的情况判断注意力是否集中，若所述目光射线没有焦点，或者所述目光射线的焦点在一定区域内停留的时间超过预设的时间范围，则判定为注意力不集中；若所述目光射线的焦点在预设的时间范围内发生移动，则判定为注意力集中。
  4. 根据权利要求2所述的智能桌，其特征在于，所述AI识别模块为完成训练的情绪识别神经网络模型，所述情绪识别神经网络模型的训练方法如下：
    获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果；
    定义损失函数，并对损失函数进行最小化操作；
    所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。
  5. 根据权利要求4所述的智能桌，其特征在于，所述对损失函数进行最小化操作包括：
    根据神经网络的所有权重和损失函数，采用反向传播法得到损失函数的梯度；
    根据所述梯度，采用随机梯度下降法，更新神经网络的权重；
    将更新的权重进行预设次数的迭代，完成对损失函数的最小化操作。
  6. 根据权利要求2所述的智能桌，其特征在于，所述本地提示装置包括控制模块与执行模块，所述控制模块与所述AI识别模块的输出端电连接，所述本地提示装置能够根据所述AI识别模块输出的注意力集中状态识别结果受到触发包括：
    所述控制模块接收所述AI识别模块输出的注意力集中状态识别结果，并统计识别得到注意力集中的状态结果与注意力不集中的状态结果的数量，若注意力集中的状态结果所占比例低于预设的比例阈值，则所述控制模块触发所述执行模块发出提示信号，所述提示信号包括声、光和/或振动信号。
  7. 根据权利要求2所述的智能桌，其特征在于，所述智能桌还包括下载有第二客户端的终端，所述第二客户端与所述第一客户端能够通过后台服务器进行消息传输；
    所述第二客户端能够接收第一客户端发送的本地提示请求，下载有所述第二客户端的终端根据所述本地提示请求触发本地提示装置发出提示信号，所述提示信号包括声、光和/或振动信号。
  8. 一种状态监控系统服务器，其特征在于，所述服务器接收通过无线通信方式上传的图像，所述服务器包括AI识别模块及处理器，所述AI识别模块用于识别接收的图像中人脸处于注意力集中状态或注意力不集中状态，所述处理器用于统计所述AI识别模块的识别结果，所述服务器将接收的图像和/或所述处理器的统计结果通过无线通信方式发送至相应的第一客户端。
  9. 一种基于情绪识别的状态监控方法，其特征在于，包括以下步骤：
    获取待监控对象的视频信息，并将视频按时序提取一个或多个帧图像；
    将所述帧图像顺序输入预训练的情绪识别神经网络模型，所述情绪识别神经网络模型用于识别帧图像中人脸处于注意力集中状态或注意力不集中状态；
    接收所述情绪识别神经网络模型输出的对应于每个帧图像的识别结果。
  10. 根据权利要求9所述的状态监控方法，其特征在于，在将视频按时序提取一个或多个帧图像之后还包括：
    根据人脸识别算法，提取所述帧图像中的人脸区域，生成对应的人脸区域图像。
  11. 根据权利要求9所述的状态监控方法，其特征在于，所述情绪识别神经网络模型的训练方法如下：
    获取训练样本，所述训练样本包括若干个人脸图像样本及对应注意力集中状态的预标定结果；
    定义损失函数，并对损失函数进行最小化操作；
    所述情绪识别神经网络模型根据所述训练样本及完成最小化的损失函数进行自学习，直至识别结果的准确率达到预设的百分比阈值，则得到完成训练的情绪识别神经网络模型。
  12. 根据权利要求9所述的状态监控方法，其特征在于，所述接收所述情绪识别神经网络模型输出的对应于每个帧图像的识别结果之后还包括：统计识别得到注意力集中的状态结果所占比例；及/或
    将统计结果发送至施行监控方的第一客户端；或者，
    若所述注意力集中的状态结果所占比例低于预设的比例阈值，则向施行监控方的第一客户端发送提示消息，或者向被监控方的第二客户端发送提示消息；或者，
    接收施行监控方的第一客户端发送的消息或请求，并将所述消息或请求转发至被监控方的第二客户端。
PCT/CN2019/099957 2018-08-11 2019-08-09 具有状态监控功能的智能桌、监控系统服务器及监控方法 WO2020034902A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/267,501 US11836631B2 (en) 2018-08-11 2019-08-09 Smart desk having status monitoring function, monitoring system server, and monitoring method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810911166 2018-08-11
CN201810911166.7 2018-08-11

Publications (1)

Publication Number Publication Date
WO2020034902A1 true WO2020034902A1 (zh) 2020-02-20

Family

ID=68149255

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/099957 WO2020034902A1 (zh) 2018-08-11 2019-08-09 具有状态监控功能的智能桌、监控系统服务器及监控方法

Country Status (3)

Country Link
US (1) US11836631B2 (zh)
CN (2) CN110334697B (zh)
WO (1) WO2020034902A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111966316A (zh) * 2020-08-25 2020-11-20 西安万像电子科技有限公司 图像数据的显示方法及装置、图像数据显示系统
CN112230760A (zh) * 2020-09-17 2021-01-15 淮南师范学院 一种基于用户操作及生物特征相结合的分析系统和方法
CN112699774A (zh) * 2020-12-28 2021-04-23 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113139439A (zh) * 2021-04-06 2021-07-20 广州大学 一种基于人脸识别的在线学习专注度评价方法及装置
CN113824967A (zh) * 2021-11-23 2021-12-21 环球数科集团有限公司 一种基于深度学习的视频压缩方法
CN115841651A (zh) * 2022-12-13 2023-03-24 广东筠诚建筑科技有限公司 基于计算机视觉与深度学习的施工人员智能监测系统

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11922314B1 (en) * 2018-11-30 2024-03-05 Ansys, Inc. Systems and methods for building dynamic reduced order physical models
JP7240916B2 (ja) * 2019-03-22 2023-03-16 ソニーセミコンダクタソリューションズ株式会社 情報処理装置、情報処理方法、および情報処理プログラム
US20210125065A1 (en) * 2019-10-25 2021-04-29 Affectiva, Inc. Deep learning in situ retraining
JP7264028B2 (ja) * 2019-12-05 2023-04-25 トヨタ自動車株式会社 情報提供システム、情報提供方法、情報端末及び情報表示方法
CN111144321B (zh) * 2019-12-28 2023-06-09 北京如布科技有限公司 专注度检测方法、装置、设备和存储介质
CN111414892B (zh) * 2020-04-09 2023-05-12 上海盛付通电子支付服务有限公司 直播中的信息发送方法
CN111709362B (zh) * 2020-06-16 2023-08-08 百度在线网络技术（北京）有限公司 用于确定重点学习内容的方法、装置、设备及存储介质
CN111862706A (zh) * 2020-07-27 2020-10-30 联想(北京)有限公司 一种基于在线教学的信息处理方法、系统及电子设备
CN112069931B (zh) * 2020-08-20 2024-10-18 深圳数联天下智能科技有限公司 一种状态报告的生成方法及状态监控系统
CN112053224B (zh) * 2020-09-02 2023-08-18 中国银行股份有限公司 业务处理监控实现方法、装置及系统
CN112306832A (zh) * 2020-10-27 2021-02-02 北京字节跳动网络技术有限公司 用户状态响应方法、装置、电子设备及存储介质
CN112380987B (zh) * 2020-11-13 2021-12-17 圣凯诺服饰有限公司 有效学习时长统计系统
CN112613444A (zh) * 2020-12-29 2021-04-06 北京市商汤科技开发有限公司 一种行为检测的方法、装置、电子设备及存储介质
CN112863254A (zh) * 2020-12-29 2021-05-28 河南库课数字科技有限公司 一种幼儿学前教育同步移动教育装置及方法
CN113008299A (zh) * 2021-02-22 2021-06-22 河北地质大学 在线学考一体化智慧课桌系统及相应的监测方法
CN113076831A (zh) * 2021-03-23 2021-07-06 上海燊睿信息科技有限公司 基于ai的教学提升的技术实现方法
CN113259734B (zh) * 2021-06-04 2023-02-03 重庆晋才富熙科技有限公司 针对交互场景的智能导播方法、装置、终端和存储介质
CN113727074A (zh) * 2021-08-31 2021-11-30 维沃移动通信有限公司 监控信息提示方法、装置和电子设备
CN113923404A (zh) * 2021-09-28 2022-01-11 宁波书写芯忆科技有限公司 一种智能专注力的监测系统
CN114484791A (zh) * 2022-01-04 2022-05-13 青岛海尔空调器有限总公司 用于调节环境的方法及装置、空调器、存储介质
CN114234078B (zh) * 2022-01-17 2024-09-03 读书郎教育科技有限公司 一种智能学习台灯的交互装置及方法
CN114915764B (zh) * 2022-05-17 2024-04-23 润芯微科技(江苏)有限公司 一种智能ai环视监控系统及使用方法
CN115032905A (zh) * 2022-05-27 2022-09-09 青岛海尔科技有限公司 场景下的提示方法及装置、存储介质及电子装置
CN116052260B (zh) * 2023-03-24 2023-06-30 江西省气象服务中心(江西省专业气象台、江西省气象宣传与科普中心) 一种天气会商视频会议点名方法及系统
CN116682052B (zh) * 2023-04-20 2024-08-23 北京北水融智教育科技有限公司 一种基于云服务的教学服务平台
CN117279170B (zh) * 2023-10-11 2024-03-22 惠州市沃生照明有限公司 用于护眼灯的灯光调节方法及系统
CN118297306A (zh) * 2024-03-12 2024-07-05 北京宜通华瑞科技有限公司 一种基于人脸识别技术的智能生产工位屏
CN118522450A (zh) * 2024-05-11 2024-08-20 浙江永鹏家具制造有限公司 一种智能办公桌的控制方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106125614A (zh) * 2016-07-27 2016-11-16 重庆工程学院 智能书桌及其控制方法
US20170344806A1 (en) * 2016-05-27 2017-11-30 Hon Hai Precision Industry Co., Ltd. Face recognition system and face recognition method
CN107766787A (zh) * 2016-08-16 2018-03-06 深圳云天励飞技术有限公司 人脸属性识别方法、装置、终端及存储介质
CN108041819A (zh) * 2017-12-08 2018-05-18 四川逸览天下文化传播有限公司 适于家用的儿童学习智能辅助装置
CN108304779A (zh) * 2017-12-28 2018-07-20 合肥壹佰教育科技有限公司 一种学生教育管理的智能化调控方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1225375C (zh) * 2003-07-02 2005-11-02 北京交通大学 基于多特征融合的困倦驾驶检测方法
CN100373400C (zh) * 2006-03-23 2008-03-05 上海交通大学 基于识别模型的多神经网络结合的睁眼检测方法
CN203637678U (zh) * 2013-12-05 2014-06-11 长安大学 一种驾驶员注意力分散预警装置
US20180144651A1 (en) * 2015-07-13 2018-05-24 Mardoche Abdelhak Teaching method using pupil's own likeness as a virtual teacher
CN106251289A (zh) * 2016-07-21 2016-12-21 北京邮电大学 一种基于深度学习和自相似性的视频超分辨率重建方法
CN106484124B (zh) * 2016-11-14 2019-07-26 北京英贝思科技有限公司 一种目光操控方法
CN106851216B (zh) * 2017-03-10 2019-05-28 山东师范大学 一种基于人脸和语音识别的课堂行为监控系统及方法
CN107145864A (zh) * 2017-05-08 2017-09-08 湖南科乐坊教育科技股份有限公司 一种专注力评估方法及系统
CN107369341A (zh) * 2017-06-08 2017-11-21 深圳市科迈爱康科技有限公司 教育机器人
CN107194371B (zh) * 2017-06-14 2020-06-09 易视腾科技股份有限公司 基于层次化卷积神经网络的用户专注度识别方法及系统
CN107705807B (zh) * 2017-08-24 2019-08-27 平安科技（深圳）有限公司 基于情绪识别的语音质检方法、装置、设备及存储介质
CN107818310B (zh) * 2017-11-03 2021-08-06 电子科技大学 一种基于视线的驾驶员注意力检测方法
CN108171176B (zh) * 2017-12-29 2020-04-24 中车工业研究院有限公司 一种基于深度学习的地铁司机情绪辨识方法及装置
CN108805009A (zh) * 2018-04-20 2018-11-13 华中师范大学 基于多模态信息融合的课堂学习状态监测方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170344806A1 (en) * 2016-05-27 2017-11-30 Hon Hai Precision Industry Co., Ltd. Face recognition system and face recognition method
CN106125614A (zh) * 2016-07-27 2016-11-16 重庆工程学院 智能书桌及其控制方法
CN107766787A (zh) * 2016-08-16 2018-03-06 深圳云天励飞技术有限公司 人脸属性识别方法、装置、终端及存储介质
CN108041819A (zh) * 2017-12-08 2018-05-18 四川逸览天下文化传播有限公司 适于家用的儿童学习智能辅助装置
CN108304779A (zh) * 2017-12-28 2018-07-20 合肥壹佰教育科技有限公司 一种学生教育管理的智能化调控方法

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111966316B (zh) * 2020-08-25 2023-08-25 西安万像电子科技有限公司 图像数据的显示方法及装置、图像数据显示系统
CN111966316A (zh) * 2020-08-25 2020-11-20 西安万像电子科技有限公司 图像数据的显示方法及装置、图像数据显示系统
CN112230760A (zh) * 2020-09-17 2021-01-15 淮南师范学院 一种基于用户操作及生物特征相结合的分析系统和方法
CN112230760B (zh) * 2020-09-17 2022-09-20 淮南师范学院 一种基于用户操作及生物特征相结合的分析系统和方法
CN112699774A (zh) * 2020-12-28 2021-04-23 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN112699774B (zh) * 2020-12-28 2024-05-24 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113052064B (zh) * 2021-03-23 2024-04-02 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113139439A (zh) * 2021-04-06 2021-07-20 广州大学 一种基于人脸识别的在线学习专注度评价方法及装置
CN113824967B (zh) * 2021-11-23 2022-02-15 环球数科集团有限公司 一种基于深度学习的视频压缩方法
CN113824967A (zh) * 2021-11-23 2021-12-21 环球数科集团有限公司 一种基于深度学习的视频压缩方法
CN115841651B (zh) * 2022-12-13 2023-08-22 广东筠诚建筑科技有限公司 基于计算机视觉与深度学习的施工人员智能监测系统
CN115841651A (zh) * 2022-12-13 2023-03-24 广东筠诚建筑科技有限公司 基于计算机视觉与深度学习的施工人员智能监测系统

Also Published As

Publication number Publication date
US11836631B2 (en) 2023-12-05
CN110334697B (zh) 2021-08-27
CN113792625A (zh) 2021-12-14
US20210326585A1 (en) 2021-10-21
CN110334697A (zh) 2019-10-15

Similar Documents

Publication Publication Date Title
WO2020034902A1 (zh) 具有状态监控功能的智能桌、监控系统服务器及监控方法
CN105022981B (zh) 一种检测人眼健康状态的方法、装置及移动终端
KR101803081B1 (ko) 매장 관리 로봇
CN109686447B (zh) 一种基于人工智能的员工状态监测系统
CN105468147A (zh) 一种预防近视的智能设备、系统及方法
CN105205646A (zh) 自动点名系统及实现自动点名系统的方法
CN205121478U (zh) 一种预防近视的智能设备及系统
CN106235931A (zh) 控制洁面仪工作的方法及装置
WO2020020022A1 (zh) 视觉识别方法及其系统
CN109345156A (zh) 一种基于机器视觉的课堂教学质量评价系统
JP2009258175A (ja) 講義システムおよび集計システム
CN207676159U (zh) 一种家用养老管理系统
CN111443609A (zh) 一种基于物联网的实验室环境自适应调节方法
CN109934097A (zh) 一种基于人工智能的表情和心理健康管理系统
CN111967770A (zh) 基于大数据的问卷调查数据处理方法、装置及存储介质
CN111402096A (zh) 一种在线授课质量管理方法、系统、设备和介质
CN113783709A (zh) 基于会议系统的参会人员监测处理方法、装置、智能终端
CN111970471A (zh) 基于视频会议的参会人员评分方法、装置、设备及介质
CN114339149A (zh) 电子设备及学习监督方法
JP2020113981A (ja) 補聴器システム
CN117373092A (zh) 基于分布式摄像的考试监控方法、设备及存储介质
CN108492231A (zh) 基于学生行为数据的课堂效果信息的监控系统
CN108596159A (zh) 一种基于互联网教育的教学方法及系统
CN112396718A (zh) 基于ar技术的现场施工安全及质量监督研究系统
CN110012256A (zh) 一种融合视频通信与体征分析的系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19850242

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19850242

Country of ref document: EP

Kind code of ref document: A1